Module pipelines.rj_cor.utils

Utils for rj-cor

Functions

def build_redis_key(dataset_id: str, table_id: str, name: str, mode: str = 'prod')

Helper function for building a key where to store the last updated time

def compare_actual_df_with_redis_df(dfr: pandas.core.frame.DataFrame, dfr_redis: pandas.core.frame.DataFrame, columns: list) ‑> pandas.core.frame.DataFrame

Compare df from redis to actual df and return only the rows from actual df that are not already saved on redis.

def get_redis_output(redis_key, is_df: bool = False)

Get Redis output. Use get to obtain a df from redis or hgetall if is a key value pair.