Job Description
Job Responsibilities:
- Design test sets and conduct experiments for evaluating system response quality
- Develop manual and automated processes and methods for analyzing emergent integrity, security and privacy trends.
- Analyze system metrics such as threat identification per product feature and remedy pain points.
- Develop red teaming instructions or guidelines to train internal or external red teams
- Plan and lead campaign-style internal red team operations
- Communicate metrics and analysis to appropriate teams
- Determine conversational design, policy and product help guidance with appropriate cross-functional teams per red teaming feature
- This role may involve potential exposure to graphic and/or objectionable content including but not limited to graphic images, videos and writings, offensive or derogatory language, and other potential objectionable material, i.e. child exploitation, graphic violence, self-injury, animal abuse, and other content which may be considered offensive or disturbing.
Skills:
- Knowledge of regular expressions, SQL and basic scripting
- Excellent technical writing and communication skills both verbal and written.
- Critical thinking and problem-solving abilities.
- Creativity to identify vulnerabilities and develop innovative attack scenarios.
- Ability to work with graphic and objectionable content.
- Data annotation management
- SQL
- Python for data processing and data analysis
- Data visualization
- Descriptive statistics
- Dashboard experience (PowerBI, Unidash, Tableau)
- Knowledge of other languages
- Text data analysis (or NLP)
Education/Experience:
- Bachelor’s degree in linguistics, language technologies, computational linguistics, speech science, or related field.
Job Tags
Remote job,