OpenSearch Engineer / AWS & Data Pipelines

About the role

Ruby Labs is building a high‑scale payment orchestration platform that processes millions of transactions each month. We are looking for an experienced OpenSearch Engineer to own and scale our AWS OpenSearch infrastructure and the pipelines that keep data synchronized from PostgreSQL.

Key responsibilities

• Own and operate AWS OpenSearch Service in production, designing indices, mappings, analyzers, templates, shards and retention policies.
• Optimize search queries, aggregations, filtering, pagination and bulk indexing; troubleshoot slow queries, indexing bottlenecks and hot shards.
• Plan and execute safe upgrades, migrations, re‑indexing and capacity scaling.
• Build and maintain reliable PostgreSQL‑to‑OpenSearch pipelines using AWS DMS, OpenSearch Ingestion, Lambda, SQS, S3, ECS/EKS.
• Implement retries, idempotency, deduplication, dead‑letter handling and recovery mechanisms; monitor ingestion lag, throughput and data freshness.
• Develop reconciliation processes to detect and repair missing or stale documents while protecting PostgreSQL performance.
• Monitor cluster health, search and indexing latency, storage, CPU, JVM pressure and shard distribution; create CloudWatch dashboards, alerts and runbooks.
• Define backup, snapshot, restore and disaster‑recovery procedures; participate in incident response and root‑cause analysis.
• Secure access using IAM, role‑based policies, encryption and least‑privilege principles.
• Collaborate with backend, database, infrastructure and product teams to design efficient document models and define schema standards.

Required profile

• Strong production experience with OpenSearch or Elasticsearch.
• Hands‑on experience with AWS OpenSearch Service.
• Proven ability to build data pipelines between PostgreSQL and OpenSearch.
• Deep understanding of search architecture, indexing, query optimization and monitoring.
• Experience with incident handling, backup/recovery and security best practices.
• Ability to work cross‑functionally with engineering and product teams.

Required skills

• OpenSearch / Elasticsearch
• AWS OpenSearch Service
• PostgreSQL
• AWS DMS, Lambda, SQS, S3, ECS, EKS
• CloudWatch monitoring and alerting
• IAM and security configuration
• Python or Bash scripting