Module `pipelines.rj_escritorio.tweets_flamengo.tasks`

Tasks for twitter scraping.

Functions

def creat_path_tree(path): Creates a path tree.
def decode_env(value: str): Decodes a base64 value.
def fetch_last_id(q): Download last_id table from storage.
def fetch_tweets(api, q, last_id, created_at): Scrapy tweets since last_id in batchs of 100 tweets.
def get_api(): Get the Twitter API.
def get_last_id(api, q, data_path: str): Get last_id from storage table or twitter api.
def normalize_cols(df): Normalize columns names.
def save_last_id(df, q): Save the last tweet ID.
def upload_to_storage(path: str): upload data to storage