Senior Apache NiFi Data Engineer
Gegnat Inc.
Job Description & Responsibilities
Company Description Gegnat Inc., recognized with the Global Staffing Award 2024, is a specialized recruitment and staffing organization focused on connecting employers with highly qualified talent. The company partners with clients to fill critical roles efficiently, helping them save time and resources while accessing a broader and more diverse candidate pool than traditional advertising can reach. Gegnat Inc. combines external and internal market analysis, business insights, and structured execution plans to support sustainable workforce strategies. By integrating SWOT analysis, marketing and sales planning, and financial forecasting, the company offers data-driven staffing solutions tailored to client needs.
Brief Job Description
NQS is migrating closed trauma records from legacy registries (B5, TraumaBase and Trauma One formats) into a PostgreSQL
data lake for reporting in Apache Superset. The engineer will build Apache NiFi flows that pick up client-uploaded files from
Azure Blob, load them through DuckDB into PostgreSQL, and make each new facility faster to onboard. Flows are 80–90%
similar within a format, so reusable templates are the core of the work. EDI (835/837) ingestion is out of scope for now.
Key Responsibilities
- Design, build and maintain NiFi process groups that ingest, transform and load facility data end to end, keeping NiFi to
two or three processors per load and pushing heavy parsing to DuckDB SQL.
- Write and optimize SQL to parse and flatten CSV, XML, JSON, Excel and DBF files, including unions across multiple files of one dataset.
- Implement CDC, incremental loads and idempotent upserts into PostgreSQL, following the client's existing trigger-based CDC tables.
- Handle file-encoding issues, skipped rows and schema differences between facilities, logging rejected rows for review. •
Build parameterized flow templates (Parameter Contexts, NiFi Registry) so new facilities onboard quickly across the
221-facility backlog.
- Add error handling, row-count reconciliation and Slack alerting to every pipeline.
- Tune NiFi and database performance: connection pools, batching and memory for large files.
- Version-control flows and SQL and support promotion across Dev, Test and Prod; support the NiFi 1.28.1 to 2.x upgrade
if scheduled.
- Recommend improvements to credential handling and infrastructure set-up, and contribute to the client's living process
document.
- Document flows and data mappings; work daily with Cory Hubert, NQS SQL developers and the Ksolves offshore team.
Good-to-Have Skills
- Healthcare integration: EDI X12 (837, 835, 270/271), HL7, Infor Cloverleaf (future scope only)
- NiFi scripting (Groovy or Python processors) or custom Java processor development
- NiFi 2.x and CI/CD for NiFi flow deployment
- Apache Superset and OpenMetadata (client's reporting and metadata tools)
- Talend experience (client's CDC tables mirror Talend's trigger-based CDC).
About Gegnat Inc.
Required Skills
Job Details
Posted by
N/A
Posted on:
28 Oct 2026
About Gegnat Inc.
More open roles
- Senior Software Engineer (Python/Node.JS , AWS)Autodesk · India
- IT Intern - Software Development, Python & AI/MLAI SAMRAT · India
- Ansible & Python Open-source Developer for GamingHartbridge Evergrove Placements · Farrukhabad, Uttar Pradesh, India
- Senior Full Stack Developer (Python, React.js & Azure, AI)Robert Bosch · India
- Specialist System Engineering (Devops Engineer, CICD, Kubernetes, containers, React, Python, API, Deployment + Development + Automation, GitHub PipelinesAT&T · Bengaluru, Karnataka, India
- Data Scientist (Python, ML, SQL) (Bhilai)Workers Town · Raipur, Chhattisgarh, India