Senior AI Engineer
Role Overview
We are looking for a Senior AI Engineer to support the evolution, implementation, and production hardening of our client's AI-powered conversational experience.
The ideal candidate will have strong hands-on experience designing and implementing production-grade Generative AI solutions, particularly Retrieval-Augmented Generation (RAG) architectures, LLM orchestration, vector databases, and cloud-native AI services.
The role will focus on assessing and evolving an existing AI implementation, currently leveraging AWS OpenSearch and PostgreSQL/pgvector, into a secure, scalable, and reliable platform capable of delivering accurate, contextually relevant, and trustworthy responses.
A critical aspect of the position involves implementing content governance and licensing controls, ensuring that copyrighted texts and other restricted content are handled according to their licensing agreements.
The engineer will also help establish mechanisms for incorporating user-generated reviews, improving retrieval relevance, evaluating AI responses, and continuously optimizing the solution.
Key Responsibilities
AI Architecture & RAG Evolution
- Assess the existing architecture, identifying technical limitations, security risks, and opportunities for modernization.
- Design, implement, and optimize RAG pipelines leveraging AWS OpenSearch, PostgreSQL/pgvector, embeddings, and LLM orchestration frameworks.
- Improve retrieval quality through hybrid search, semantic search, metadata filtering, reranking, and context optimization.
- Develop scalable ingestion pipelines supporting structured and unstructured content from multiple sources.
- Evaluate and integrate LLM providers and models based on accuracy, latency, cost, and operational requirements.
- Define and implement architectural improvements incrementally, minimizing disruption to the existing platform.
Content Governance & Licensing
- Implement rights-aware ingestion and retrieval mechanisms to enforce content licensing restrictions.
- Design metadata-driven access controls supporting content ownership, usage permissions, subscription entitlements, and licensing expiration.
- Ensure restricted content is not exposed to unauthorized AI processing or model inference.
- Implement response validation mechanisms covering source attribution, quotation limits, and copyrighted content protection.
- Support content lifecycle management, including reindexing, content removal, and cache invalidation.
AI Security, Reliability & Scalability
- Design and implement secure AI services using AWS cloud infrastructure and established security practices.
- Introduce AI-specific guardrails addressing prompt injection, jailbreak attempts, sensitive-data exposure, and unauthorized content retrieval.
- Implement authentication, authorization, API security, rate limiting, and secure service-to-service communication.
- Optimize inference performance, token consumption, caching strategies, and operational costs.
- Establish monitoring, observability, logging, and alerting for AI workloads.
- Implement resilience mechanisms, including fallback strategies, controlled deployments, and feature flags.
AI Quality, Evaluation & Continuous Improvement
- Develop automated evaluation frameworks to measure response accuracy, groundedness, retrieval relevance, and citation correctness.
- Implement mechanisms to collect and analyze feedback on AI-generated responses.
- Design approaches for incorporating user-generated reviews, comments, and reflections as supplementary knowledge sources.
- Establish moderation, classification, and validation processes for user-generated content before making it available to RAG pipelines.
- Maintain clear separation between authoritative sources, licensed editorial content, and community-generated perspectives.
- Implement regression testing and continuous improvement processes for prompts, retrieval configurations, and model behavior.
- Collaborate with product and content stakeholders to address response quality and interpretation concerns.
Technical Leadership & Collaboration
- Provide technical guidance on AI architecture, implementation decisions, and production best practices.
- Collaborate with backend engineers, cloud engineers, product teams, and content stakeholders.
- Translate business, licensing, and product requirements into scalable technical solutions.
- Contribute to architecture documentation, technical design reviews, and engineering standards.
- Support the transition from the current implementation toward a maintainable, production-ready AI platform.
Must-Have Qualifications
- 5+ years of software engineering experience, including substantial hands-on experience developing AI/ML or Generative AI solutions.
- Proven experience implementing and operating production-grade LLM applications, particularly RAG-based systems.
- Strong proficiency in Python, including backend development, API integration, and data-processing pipelines.
- Hands-on experience with AWS OpenSearch, Elasticsearch, PostgreSQL/pgvector, or comparable vector search technologies.
- Solid understanding of embeddings, vector similarity search, hybrid retrieval, reranking, and document chunking strategies.
- Experience integrating LLM APIs and working with orchestration frameworks such as LangChain, LangGraph, LlamaIndex, or equivalent custom implementations.
- Practical experience with AWS cloud services, including secure deployment, IAM, networking, monitoring, and scalable backend infrastructure.
- Understanding of AI security, including prompt injection prevention, content filtering, access controls, and LLM guardrails.
- Experience designing and implementing AI evaluation and testing strategies.
- Strong knowledge of REST APIs, microservices, asynchronous processing, and distributed system design.
- Ability to independently assess existing architectures, recommend improvements, and implement production-ready solutions.
- Strong communication skills and ability to work directly with technical and non-technical stakeholders.
Nice-to-Have Qualifications
- Experience with Amazon Bedrock, including managed foundation models, guardrails, and model evaluation.
- Experience implementing document-level authorization, digital rights management, or licensing-aware content retrieval.
- Familiarity with publishing, digital content platforms, content management systems, or subscription-based products.
- Experience implementing user-generated content ingestion, moderation, and community-driven knowledge enrichment.
- Knowledge of advanced RAG techniques, including query rewriting, multi-stage retrieval, contextual compression, and agentic retrieval.
- Experience with LLM observability and evaluation tools such as LangSmith, Langfuse, Ragas, DeepEval, or similar frameworks.
- Familiarity with LLMOps/MLOps practices, including CI/CD, automated evaluation, and model versioning.
- Experience optimizing LLM inference costs, latency, and caching strategies.
- Familiarity with privacy, data governance, and responsible AI principles.
- Experience with high-traffic consumer-facing AI applications.
Expected Outcomes
The successful candidate will contribute to:
- Architecture assessment and modernization: A clear understanding of the existing implementation and a practical roadmap for improving its architecture.
- Rights-aware RAG implementation: Secure ingestion and retrieval processes that respect content licensing and authorization requirements.
- Production hardening: Improved security, scalability, resilience, observability, and operational efficiency.
- Response quality improvement: Measurable improvements in retrieval relevance, answer groundedness, and citation accuracy.
- Community content integration: A controlled approach for incorporating user-generated reviews as supplementary knowledge.
- Sustainable AI engineering practices: Automated evaluation, regression testing, technical documentation, and continuous improvement processes.
Remote from Brazil or Argentina. Full-time, long-term engagement. The selection process runs in English.