Funções úteis no tratamento de dados de satélite

Functions

def choose_file_to_download(storage_files_path: list, base_path: str, redis_files: str, ref_filename=None)

We can treat only one file each run, so we will first eliminate files that were already trated and saved on redis and keep only the first one from this partition

def convert_julian_to_conventional_day(year: int, julian_day: int)

Convert julian day to conventional

Parameters year (int): 2022 (20222901900203) julian_day (int): 290 (20222901900203)

Returns date_save (str): 19 (20222901900203)

def converte_timezone(datetime_save: str) ‑> str

Get UTC date-hour on 'YYYYMMDD HHmm' format and returns im the same format but on São Paulo timezone.

def create_and_save_image(data: xarray.core.dataarray.DataArray, info: dict, variable) ‑> pathlib.Path

Create image from xarray ans save it as png file.

def download_blob(bucket_name: str, source_blob_name: str, destination_file_name: Union[str, pathlib.Path], mode: str = 'prod')

Downloads a blob from the bucket. Mode needs to be "prod" or "staging"

The ID of your GCS bucket

bucket_name = "your-bucket-name"

The ID of your GCS object

source_blob_name = "storage-object-name"

The path to which the file should be downloaded

destination_file_name = "local/path/to/file"

def extract_julian_day_and_hour_from_filename(filename: str)

Extract julian day and hour from satelite filename

Parameters filename (str): 'OR_ABI-L2-TPWF-M6_G16_s20222901900203_e20222901909511_c20222901911473.nc'

Returns year (int): 2022 (20222901900203) julian_day (int): 290 (20222901900203) hour_utc (str): 1900 (20222901900203)

def get_blob_with_prefix(bucket_name: str, prefix: str, mode: str = 'prod') ‑> str

Lists all the blobs in the bucket that begin with the prefix. This can be used to list all blobs in a "folder", e.g. "public/". Mode needs to be "prod" or "staging"

def get_files_from_aws(partition_path)

Get all available files from aws that is inside the partition path

def get_files_from_gcp(partition_path)

Get all available files from gcp that is inside the partition path

def get_info(path: str) ‑> dict

Getting Information From the File Name (Time, Date,

Product Type, Variable and Defining the CMAP)

def get_variable_values(dfr: pandas.core.frame.DataFrame, variable: str) ‑> xarray.core.dataarray.DataArray

Convert pandas dataframe to a matrix with latitude on rows, longitudes on columns and the correspondent values on a xarray DataArray

def read_netcdf(file_path: str) ‑> pandas.core.frame.DataFrame

Function to extract data from NetCDF file and convert it to pandas DataFrame with the name of columns the same as the variable saved on the filename

def remap_g16(path: str, extent: list, product: str, variable: list)

the GOES-16 image is reprojected to the rectangular projection in the extent region. If netcdf file has more than one variable remap function will save each variable in a different netcdf file inside temp/ folder.

def save_data_in_file(product: str, variable: list, datetime_save: str, mode_redis: str = 'prod')

Read all nc or tif files and save them in a unique file inside a partition

def upload_image_to_api(info: dict, save_image_path: pathlib.Path)

Upload image to api