Data Engineer III - Crystal Equation Corporation : Job Details

Data Engineer III Remote Summary: The CW will support efforts to run CSAM filtering on image and video datasets leveraged by FAIR research teams. The goal is to proactively mitigate potential risks associated with these datasets. Responsibilities: Preprocessing: converting original datasets into a format that can be consumed by CSAM filtering pipelines. Filtering: running CSAM filtering using Integrity's pipeline or Spidermate's pipeline. Post-processing: consuming filtering results to filter videos/images in the original datasets, repackaging, and re-ingestion. Skills: Technical skills: Experience building and maintaining data pipelines, knowledge of data storage solutions; Data Management: Data preprocessing and cleaning, transformation and formatting, data quality control and validation Communication - effective communication skills to collaborate with stakeholders and team members Strongly prefer C - Ws with prior experience supporting data efforts at our company, as this would ena...Data Engineer, Engineer, Processing, Technology, Staffing, Data, Pipeline