Delivery Manager AI Observability - Warsaw - Associate

• Delivery Manager AI Observability - Warsaw - Associate
• location_on

Warsaw, Mazowieckie, Poland

share

Share

bookmark_border

Save

Opportunity Overview

sitemap_outline

CORPORATE TITLE

Associate

language

OFFICE LOCATION(S)

Warsaw

assignment

JOB FUNCTION

Project Management

account_balance

DIVISION

Engineering Division

Apply

As a Senior AI Guardrail Evaluation Expert & Project Manager, you will play a pivotal role in safeguarding the integrity and trustworthiness of our AI systems. You will lead the charge in evaluating and refining the guardrails that protect our models from generating harmful, biased, or unfaithful content. This is a unique chance to combine your deep technical expertise with your leadership skills, working within a dynamic, hybrid environment to define best practices in AI ethics and safety for the financial industry.

You’ll do

Lead the comprehensive evaluation and continuous refinement of AI guardrails to ensure robust detection and mitigation of harmful content and behaviours.

Develop and improve sophisticated strategies for evaluating guardrails against

Profane, Toxic, and Adult Content

• Bias detection and mitigation in AI outputs
• Illegal and Harmful Content generation
• Misuse of Personal Data (PII) and other sensitive information
• Hallucination and the generation of unfaithful or factually incorrect content
• Drive the planning, execution, and meticulous documentation of complex AI guardrail evaluation projects, ensuring adherence to timelines, resources, and quality deliverables.
• Identify, onboard, train, and motivate a team of skilled human annotators, providing clear guidance and fostering a high-performance environment.
• Leverage strong linguistic understanding to guide annotation efforts, refine guardrail performance, and ensure high-quality human feedback.
• Collaborate cross-functionally with AI researchers, engineers, product managers, and legal teams to integrate ethical considerations throughout the AI development lifecycle.
• You'll bring
• A proven track record in AI/ML ethics, content moderation, linguistic analysis, or a related field, with significant project leadership experience.
• Deep practical expertise in evaluating and improving AI guardrails on an evaluation platform like Braintrust, specifically addressing the categories mentioned above.
• Proficiency with version control systems (e.g., GitHub), collaboration platforms (e.g., Confluence), and data science environments (e.g., JupyterHub).
• Scripting and automation skills, particularly with Python, for data analysis and workflow optimization.
• Demonstrated ability to plan, execute, and meticulously document complex projects, managing timelines, resources, and deliverables effectively.
• Exceptional leadership skills, with experience in building, motivating, and managing high-performing teams, especially human annotators.
• Superior linguistic understanding and analytical capabilities to guide complex annotation tasks and interpret nuanced AI behaviours.
• Excellent communication, interpersonal, and problem-solving skills, with the ability to articulate complex technical concepts.

ABOUT GOLDMAN SACHS

At Goldman Sachs, we commit our people, capital and ideas to help our clients, shareholders and the communities we serve to grow. Founded in 1869, we are a leading global investment banking, securities and investment management firm. Headquartered in New York, we maintain offices around the world.

We believe who you are makes you better at what you do. We're committed to fostering and advancing diversity and inclusion in our own workplace and beyond by ensuring every individual within our firm has a number of opportunities to grow professionally and personally, from our training and development opportunities and firmwide networks to benefits, wellness and personal finance offerings and mindfulness programs. Learn more about our culture, benefits, and people at GS.com/careers.

We’re committed to finding reasonable accommodations for candidates with special needs or disabilities during our recruiting process. Learn more: https://www.goldmansachs.com/careers/footer/disability-statement.html

© The Goldman Sachs Group, Inc., 2023. All rights reserved.

Goldman Sachs is an equal opportunity employer and does not discriminate on the basis of race, color, religion, sex, national origin, age, veterans status, disability, or any other characteristic protected by applicable law.

We Offer Best-In-Class Benefits

Healthcare & Medical Insurance

We offer a wide range of health and welfare programs that vary depending on office location. These generally include medical, dental, short-term disability, long-term disability, life, accidental death, labor accident and business travel accident insurance.

Holiday & Vacation Policies

We offer competitive vacation policies based on employee level and office location. We promote time off from work to recharge by providing generous vacation entitlements and a minimum of three weeks expected vacation usage each year.

Financial Wellness & Retirement

We assist employees in saving and planning for retirement, offer financial support for higher education, and provide a number of benefits to help employees prepare for the unexpected. We offer live financial education and content on a variety of topics to address the spectrum of employees’ priorities.

Health Services

We offer a medical advocacy service for employees and family members facing critical health situations, and counseling and referral services through the Employee Assistance Program (EAP). We provide Global Medical, Security and Travel Assistance and a Workplace Ergonomics Program. We also offer state-of-the-art on-site health centers in certain offices.

Fitness

To encourage employees to live a healthy and active lifestyle, some of our offices feature on-site fitness centers. For eligible employees we typically reimburse fees paid for a fitness club membership or activity (up to a pre-approved amount).

Child Care & Family Care

We offer on-site child care centers that provide full-time and emergency back-up care, as well as mother and baby rooms and homework rooms. In every office, we provide advice and counseling services, expectant parent resources and transitional programs for parents returning from parental leave. Adoption, surrogacy, egg donation and egg retrieval stipends are also available.

Benefits at Goldman Sachs

Read more about the full suite of class-leading benefits our firm has to offer.

Learn More