Module pipelines.utils.dump_datario.tasks

General purpose tasks for dumping database data.

Functions

def get_datario_geodataframe(url: str, path: Union[str, pathlib.Path], wait=None)

" Save a CSV from data.rio API

Parameters

• url (str): URL of the data.rio API
• path (Union[str, Path]): Local path to save the file
• wait (Optional[Any]): Prefect task wait parameter (default: None)

def transform_geodataframe(file_path: Union[str, pathlib.Path], batch_size: int = 50000, geometry_column: str = 'geometry', convert_to_crs_4326: bool = False, geometry_3d_to_2d: bool = False, wait=None)

" Transform a CSV from data.rio API

Parameters

• file_path (Union[str, Path]): Path to the geojson file to be transformed.
• batch_size (int): Number of rows to process at once.
• geometry_column (str): Column containing the geometry data.
• convert_to_crs_4326 (bool): Convert the geometry data to the crs 4326 projection.
• geometry_3d_to_2d (bool): Convert the geometry data from 3D to 2D.
• wait (None): Prefect task wait parameter (default: None)