Engineering / Arlington, Virginia, United States, Remote Work

Senior Data Engineer, Kafka

Reducing months of backward-looking manual spreadsheet inputs to instant visualizations with continuous monitoring, the Interos platform helps the world’s companies reduce risk, avoid disruptions, and achieve dramatically superior resilience. Businesses can uncover game-changing opportunities that radically change the way they see, learn and profit from their relationships.

Apply

Interos is the operational resilience company — reinventing how companies manage their supply chains and business relationships — through a breakthrough SaaS platform that uses artificial intelligence to model and transform the ecosystems of complex businesses into a living global map, down to any single supplier, anywhere.

Reducing months of backward-looking manual spreadsheet inputs to instant visualizations with continuous monitoring, the Interos platform helps the world’s companies reduce risk, avoid disruptions, and achieve dramatically superior resilience. Businesses can uncover game-changing opportunities that radically change the way they see, learn and profit from their relationships.

Based in Washington, DC, Interos serves global clients with business-critical, interdependent relationships. The fast-growing private company is led by CEO Jennifer Bisceglie and supported by investors Venrock and Kleiner Perkins. For more information, visit www.interos.ai.

The Opportunity:

Looking for expert technical leadership to help us design, build, and iterate our next generation platform.  As someone who has successfully implemented a significant Kafka based machine learning or data analytics pipeline you can help us make the right choices and avoid mistakes as we build a scalable, reliable, and fully automated supply chain risk management system for our exponentially growing customer base.  You will be working with a collaborative team of highly skilled and constantly learning software engineers and data scientists who value good ideas and the ability to create quality solutions.

 

Primary Purpose of the Position:

The Senior Data Engineer, Kafka will guide and vet our design decisions around Kafka and our next-gen architecture.  You will help us optimize our Kafka usage and implementation and mentor other engineers on best practices around Kafka and event streaming.

 

Essential Functions/Duties:

  • Kafka topic topology design.
  • Kafka message schema evolution.
  • Write Kafka producers and consumers in Python.
  • Guide data and machine learning engineers on best practices regarding Kafka.
  • Design and build near real-time data analytics pipelines.
  • Pitch in wherever needed to meet team goals and deliver a quality product.

 

Minimum Qualifications:

  • 5+ years of hands-on experience with Kafka Cluster management (Apache Kafka or Confluent Platform).
  • Overall experience in Kafka as a solution including the various part of the streaming platform and different components.
  • Experience in open source and confluent Kafka, zookeepers, Kafka connect, schema registry, KSQL, Rest proxy and Kafka Control. center. Experience in Confluent Replicator is appreciated.
  • Hands on experience in Kafka topic sizing, performance monitoring, broker security, topic security, consumer/producer access management (ACL).
  • Knowledge of Kafka API.
  • Experience in Message Serialization/Storage Formats such as AVRO, Parquet, Protobufs, etc.
  • Strong Python coding skills.
  • Ability to communicate clearly and willingness to share knowledge and collaborate.
  • Minimum education level: Bachelor’s Degree, Computer Science or related field or equivalent.

 

Preferred Qualifications:

  • Experience in JAVA and understanding the difference between the JAVA and Python.
  • Experience in performance improvement for distributed processing.
  • Strong SLDC experience along with Agile methodologies and DevOps.
  • Experience in integration with real-time systems, analytics, and data warehouses.
  • Experience in PostgresSQL, SQL, Snowflake.
  • Experience in Spark and/or Databricks batching and streaming.
  • Experience in building Microservices using libraries like FastApi or gRPC.
  • Comfortable working with Docker and Kubernetes.

Additional Information:

  • Supervisory Responsibility: This position has no supervisory responsibilities.
  • Travel Requirements: This position may require up to 10% travel. Frequent travel is outside the local area and overnight.
  • Work Environment:  This job operates in a professional office environment. This role routinely uses standard office equipment such as computers, phones, printers.
  • Physical Demands: This is largely a sedentary role. Physical requirements include occasional lifting/carrying of 5 pounds; visual acuity, speech and hearing; hand and eye coordination and manual dexterity necessary to operate a computer keyboard and basic office equipment. Subject to sitting, standing, reaching, walking, twisting, and kneeling to perform the essential functions. Working conditions are primarily inside an office environment.
  • FLSA Status: Exempt

 

BENEFITS:

  • Comprehensive Health & Wellness package (Medical, Dental and Vision)
  • 10 Paid Holiday Days Off
  • Flexible Paid Time Off (FTO)
  • 401 (k) Employer Matching
  • Stock Options
  • Career advancement opportunities
  • Casual Dress
  • On-site gym and dedicated Peloton room at headquarters
  • Company Events (Sports Games, Fitness Competitions, Birthday Celebrations, Contests, Happy Hours)
  • Annual company party
  • Employee Referral Program

Interos is proud to be an Equal Opportunity Employer and will consider all qualified applicants without regard to race, color, age, religion, sex, sexual orientation, gender identity, genetic information, national origin, disability, protected veteran status or any other classification protected by law.

If you are a candidate in need of assistance or an accommodation in the application process, please contact [email protected]

Career image

Apply

Interos is proud to be an Equal Opportunity Employer and will consider all qualified applicants without regard to race, color, age, religion, sex, sexual orientation, gender identity, genetic information, national origin, disability, protected veteran status or any other classification protected by law.

If you are a candidate in need of assistance or an accommodation in the application process, please contact HR@interos.com

Apply

Ensure Operational Resilience

Request Contact

Build operational resiliency into your extended supply chain:

  • 889 compliance – ensure market access
  • Data sharing with 3rd parties and beyond – protect reputation
  • Concentration risk – ensure business continuity
  • Cyber breaches – assess potential exposure
  • Unethical labor – avoid reputational harm
  • On-boarding and monitoring suppliers – save time and money