Data Engineer III - Crystal Equation Corporation : Job Details

Data Engineer III Remote Summary: 4-5 years of experience in data processing and management, with a strong background in building and maintaining data pipelines, data preprocessing and cleaning, and data quality control. Prior experience supporting data efforts at our company is highly preferred. The CW will support efforts to run CSAM filtering on image and video datasets leveraged by FAIR research teams. The goal is to proactively mitigate potential risks associated with these datasets. Responsibilities: Preprocessing: converting original datasets into a format that can be consumed by CSAM filtering pipelines. Filtering: running CSAM filtering using Integrity's pipeline or Spidermate's pipeline. Post-processing: consuming filtering results to filter videos/images in the original datasets, repackaging, and re-ingestion. Skills: Technical skills: Experience building and maintaining data pipelines, knowledge of data storage solutions; Data Management: Data preprocessing and cleaning, tr...Data Engineer, Engineer, Processing, Technology, Staffing, Data, Pipeline