View job listing
Apply to Microsoft Azure Data Engineer — Azure Data Platform and Reporting Remediation
Position: Microsoft Azure Data Engineer — Azure Data Platform and Reporting Remediation
Location: Remote — United States only
Employment Type: Full-time contract
Estimated Duration: Approximately two months
Reports To: Managing Director of Technical Services
Experience Level: Senior
Overview
Our client, an intelligent automation and AI consultancy, is seeking an experienced Microsoft Azure Data Engineer to stabilize, remediate, and extend its internal Azure data platform supporting executive and Client Principal reporting.
The environment integrates data from Upland PSA and HubSpot CRM into Azure SQL Database using Azure Data Factory pipelines and T-SQL stored procedures. Power BI provides executive reporting on revenue, backlog, utilization, forecast, and margin. Additional integrations currently operate through Workato, and QuickBooks financial data has not yet been integrated into the reporting platform.
The Data Engineer will independently diagnose existing SQL logic defects, correct historical data modeling issues, improve client master data alignment, repair reporting calculations, and migrate legacy integrations into Azure Data Factory.
This is a tactical, delivery-focused engagement requiring an individual capable of understanding existing production systems, translating complex business rules into reliable data logic, and delivering tested, documented solutions with minimal supervision.
Objectives
1. Restore Accuracy to Executive Financial and Operational Reporting
Within the first three to four weeks, diagnose and remediate defects affecting forecasted hours, utilization, staffing capacity, project costs, margin, and week-over-week backlog reporting. Analyze existing Azure SQL stored procedures, MERGE/upsert logic, Azure Data Factory pipelines, and Power BI semantic models to isolate root causes. Develop reconciliation queries and validation procedures that demonstrate alignment with Upland PSA source data within agreed tolerances. Success will be measured by validated reporting accuracy across representative active projects and the ability of Client Principals to identify project-level backlog and forecast changes between consecutive weeks.
2. Establish Historically Accurate Revenue Attribution and Client Master Data
Within the first four to six weeks, design and implement data model enhancements that preserve historical revenue attribution, establish consistent client identifiers, and differentiate end clients from billing partners. Implement appropriate effective-dated employee assignment dimensions, historical snapshots, client crosswalks, and reporting relationships within the existing Azure SQL environment. Coordinate business-rule validation with Finance and other stakeholders, including the treatment of late time entries and post-close adjustments. Success will be measured by preservation of closed-period revenue following employee team changes, consistent active-client identification across HubSpot and Upland, and accurate end-client reporting for partner-billed engagements.
3. Migrate Legacy Workato Integrations into Azure Data Factory
By the end of the estimated two-month engagement, rebuild the remaining in-scope Workato integrations, estimated at four or fewer recipes, as production-ready Azure Data Factory pipelines. Document existing integration logic, dependencies, authentication mechanisms, scheduling, transformation requirements, and error-handling processes. Conduct parallel validation runs and implement appropriate monitoring, failure handling, and operational controls before supporting cutover. Success will be measured by equivalent or improved functionality, consistent output between Workato and Azure Data Factory, and readiness to retire the legacy integrations.
4. Evaluate and Recommend a Sustainable QuickBooks Integration Architecture
Within the first four to six weeks, evaluate the technical and commercial options for integrating QuickBooks financial information into *** Azure and Power BI reporting environment. Compare a native Azure Data Factory implementation using the QuickBooks API against appropriate third-party connector alternatives. Assess costs, technical complexity, security, SOC 2 implications, maintainability, and alignment with the planned retirement of Workato. Deliver a documented recommendation reviewed with the Managing Director of Technical Services, with implementation dependent on separate scope confirmation.
5. Deliver Secure, Documented, and Maintainable Production Solutions
Throughout the engagement, ensure all SQL, pipeline, and Power BI semantic model changes follow *** Azure DevOps source control, peer-review, change management, and SOC 2 requirements. Develop clear technical documentation covering data lineage, transformation logic, business rules, validation results, operational dependencies, and maintenance considerations. Coordinate knowledge transfer with internal technical resources and provide specifications for any Power BI report or visual changes to be completed by *** personnel. Success will be measured by documented acceptance of each workstream, reviewed production changes, completed knowledge transfer, and the ability of internal teams to maintain the delivered solutions.
Subtasks
1. Assess the Existing Azure Data Platform and Prioritize Remediation
Develop a prioritized remediation plan covering workstream dependencies, technical risks, validation procedures, and delivery milestones. Measure success through completion of an agreed-upon assessment and delivery plan.
2. Remediate Forecast, Utilization, Cost, Margin, and Backlog Calculations
Validate corrections using reconciliation queries that demonstrate source-system alignment within agreed tolerances and accurate project-level comparisons between consecutive weeks.
3. Implement Point-in-Time Historical Revenue Attribution
Redesign the applicable data model and upsert logic to prevent employee role or team changes from rewriting historical revenue attribution. Implement appropriate Type 2 slowly changing dimensions, effective-dated employee assignments, and/or monthly period-close snapshot tables that cannot be overwritten by routine daily ingestion.
4. Resolve Client Master Data and Partner-Billed Revenue Reporting
Design and implement a consistent unique client identifier across HubSpot and Upland PSA, including the necessary HubSpot custom property and Azure SQL historical crosswalk. Backfill historical client records and produce an exception report identifying any unmatched records requiring resolution. Modify the reporting data model to distinguish end clients from billing partners such as UiPath and Celonis. Validate that every active client carries a consistent identifier across both systems and that partner-billed revenue reports under the end client by default, with billing partner available as a reporting dimension.
5. Evaluate QuickBooks Integration Options
investigate supported approaches for bringing QuickBooks financial data into the Azure SQL and Power BI reporting environment. Assess native API integration using Azure Data Factory and compare relevant third-party connectors, including authentication, extraction limitations, operating costs, security controls, and ongoing maintenance requirements. Evaluate each option against *** SOC 2 obligations and strategy to retire Workato. Deliver a written recommendation reviewed by the Managing Director of Technical Services; production implementation will require confirmation that it is within the engagement scope.
6. Complete Workato Migration, Production Validation, and Knowledge Transfer
During the engagement and no later than its agreed completion date, inventory, document, and rebuild the remaining in-scope Workato integrations using Azure Data Factory. Implement equivalent or improved scheduling, parameterization, authentication, monitoring, failure handling, and incremental data processing where applicable. Conduct parallel runs, reconcile outputs, obtain appropriate change approvals, and support cutover readiness for retiring Workato. Complete technical documentation and knowledge transfer across all seven original workstreams, with success measured against their respective acceptance criteria.
Requirements
Preferred Qualifications
Location: Remote — United States only
Employment Type: Full-time contract
Estimated Duration: Approximately two months
Reports To: Managing Director of Technical Services
Experience Level: Senior
Overview
Our client, an intelligent automation and AI consultancy, is seeking an experienced Microsoft Azure Data Engineer to stabilize, remediate, and extend its internal Azure data platform supporting executive and Client Principal reporting.
The environment integrates data from Upland PSA and HubSpot CRM into Azure SQL Database using Azure Data Factory pipelines and T-SQL stored procedures. Power BI provides executive reporting on revenue, backlog, utilization, forecast, and margin. Additional integrations currently operate through Workato, and QuickBooks financial data has not yet been integrated into the reporting platform.
The Data Engineer will independently diagnose existing SQL logic defects, correct historical data modeling issues, improve client master data alignment, repair reporting calculations, and migrate legacy integrations into Azure Data Factory.
This is a tactical, delivery-focused engagement requiring an individual capable of understanding existing production systems, translating complex business rules into reliable data logic, and delivering tested, documented solutions with minimal supervision.
Objectives
1. Restore Accuracy to Executive Financial and Operational Reporting
Within the first three to four weeks, diagnose and remediate defects affecting forecasted hours, utilization, staffing capacity, project costs, margin, and week-over-week backlog reporting. Analyze existing Azure SQL stored procedures, MERGE/upsert logic, Azure Data Factory pipelines, and Power BI semantic models to isolate root causes. Develop reconciliation queries and validation procedures that demonstrate alignment with Upland PSA source data within agreed tolerances. Success will be measured by validated reporting accuracy across representative active projects and the ability of Client Principals to identify project-level backlog and forecast changes between consecutive weeks.
2. Establish Historically Accurate Revenue Attribution and Client Master Data
Within the first four to six weeks, design and implement data model enhancements that preserve historical revenue attribution, establish consistent client identifiers, and differentiate end clients from billing partners. Implement appropriate effective-dated employee assignment dimensions, historical snapshots, client crosswalks, and reporting relationships within the existing Azure SQL environment. Coordinate business-rule validation with Finance and other stakeholders, including the treatment of late time entries and post-close adjustments. Success will be measured by preservation of closed-period revenue following employee team changes, consistent active-client identification across HubSpot and Upland, and accurate end-client reporting for partner-billed engagements.
3. Migrate Legacy Workato Integrations into Azure Data Factory
By the end of the estimated two-month engagement, rebuild the remaining in-scope Workato integrations, estimated at four or fewer recipes, as production-ready Azure Data Factory pipelines. Document existing integration logic, dependencies, authentication mechanisms, scheduling, transformation requirements, and error-handling processes. Conduct parallel validation runs and implement appropriate monitoring, failure handling, and operational controls before supporting cutover. Success will be measured by equivalent or improved functionality, consistent output between Workato and Azure Data Factory, and readiness to retire the legacy integrations.
4. Evaluate and Recommend a Sustainable QuickBooks Integration Architecture
Within the first four to six weeks, evaluate the technical and commercial options for integrating QuickBooks financial information into *** Azure and Power BI reporting environment. Compare a native Azure Data Factory implementation using the QuickBooks API against appropriate third-party connector alternatives. Assess costs, technical complexity, security, SOC 2 implications, maintainability, and alignment with the planned retirement of Workato. Deliver a documented recommendation reviewed with the Managing Director of Technical Services, with implementation dependent on separate scope confirmation.
5. Deliver Secure, Documented, and Maintainable Production Solutions
Throughout the engagement, ensure all SQL, pipeline, and Power BI semantic model changes follow *** Azure DevOps source control, peer-review, change management, and SOC 2 requirements. Develop clear technical documentation covering data lineage, transformation logic, business rules, validation results, operational dependencies, and maintenance considerations. Coordinate knowledge transfer with internal technical resources and provide specifications for any Power BI report or visual changes to be completed by *** personnel. Success will be measured by documented acceptance of each workstream, reviewed production changes, completed knowledge transfer, and the ability of internal teams to maintain the delivered solutions.
Subtasks
1. Assess the Existing Azure Data Platform and Prioritize Remediation
Develop a prioritized remediation plan covering workstream dependencies, technical risks, validation procedures, and delivery milestones. Measure success through completion of an agreed-upon assessment and delivery plan.
2. Remediate Forecast, Utilization, Cost, Margin, and Backlog Calculations
Validate corrections using reconciliation queries that demonstrate source-system alignment within agreed tolerances and accurate project-level comparisons between consecutive weeks.
3. Implement Point-in-Time Historical Revenue Attribution
Redesign the applicable data model and upsert logic to prevent employee role or team changes from rewriting historical revenue attribution. Implement appropriate Type 2 slowly changing dimensions, effective-dated employee assignments, and/or monthly period-close snapshot tables that cannot be overwritten by routine daily ingestion.
4. Resolve Client Master Data and Partner-Billed Revenue Reporting
Design and implement a consistent unique client identifier across HubSpot and Upland PSA, including the necessary HubSpot custom property and Azure SQL historical crosswalk. Backfill historical client records and produce an exception report identifying any unmatched records requiring resolution. Modify the reporting data model to distinguish end clients from billing partners such as UiPath and Celonis. Validate that every active client carries a consistent identifier across both systems and that partner-billed revenue reports under the end client by default, with billing partner available as a reporting dimension.
5. Evaluate QuickBooks Integration Options
investigate supported approaches for bringing QuickBooks financial data into the Azure SQL and Power BI reporting environment. Assess native API integration using Azure Data Factory and compare relevant third-party connectors, including authentication, extraction limitations, operating costs, security controls, and ongoing maintenance requirements. Evaluate each option against *** SOC 2 obligations and strategy to retire Workato. Deliver a written recommendation reviewed by the Managing Director of Technical Services; production implementation will require confirmation that it is within the engagement scope.
6. Complete Workato Migration, Production Validation, and Knowledge Transfer
During the engagement and no later than its agreed completion date, inventory, document, and rebuild the remaining in-scope Workato integrations using Azure Data Factory. Implement equivalent or improved scheduling, parameterization, authentication, monitoring, failure handling, and incremental data processing where applicable. Conduct parallel runs, reconcile outputs, obtain appropriate change approvals, and support cutover readiness for retiring Workato. Complete technical documentation and knowledge transfer across all seven original workstreams, with success measured against their respective acceptance criteria.
Requirements
- Professional Experience: Minimum five years of hands-on professional data engineering experience, including at least three years working with the Microsoft Azure data stack.
- Azure Data Factory: Demonstrated production experience building and maintaining Azure Data Factory pipelines, including REST APIs, parameterization, incremental loads, scheduling, monitoring, and failure handling.
- Advanced T-SQL and Azure SQL: Strong hands-on experience with stored procedures, MERGE/upsert logic, window functions, complex SQL debugging, performance tuning, and remediation of legacy code written by other engineers.
- Historical Data Modeling: Proven implementation of Type 2 slowly changing dimensions, effective-dated dimensions, and periodic snapshot tables that preserve historical reporting accuracy.
- Dimensional Modeling: Demonstrated experience developing star schemas, conformed dimensions, master data crosswalks, and reporting-oriented data models.
- Power BI and DAX: Ability to trace reported metrics to underlying source data, modify semantic model relationships and measures, develop DAX calculations, and validate reporting outputs.
- SaaS API Integration: Hands-on experience integrating SaaS platforms through REST APIs, including OAuth authentication and pagination.
- Professional Services Reporting: Working knowledge of billable utilization, staffing capacity, backlog, forecast versus actuals, revenue attribution, and project margin.
- Independent Delivery: Demonstrated ability to independently diagnose technical problems, work with Finance and executive stakeholders, translate business requirements into data logic, and deliver documented solutions.
- Source Control: Experience using Git-based source control for database and pipeline code, preferably Azure DevOps.
Preferred Qualifications
- Experience with Upland PSA or comparable professional services automation platforms such as Kantata, Certinia, or Planview.
- Experience working with HubSpot CRM APIs, data models, and custom properties.
- Experience extracting financial data from the QuickBooks Online API or evaluating financial data integration connectors.
- Experience migrating integrations from Workato or another iPaaS platform into Azure Data Factory.
- Experience implementing CI/CD for Azure Data Factory and SQL database deployments through Azure DevOps.
- Prior experience supporting consulting or professional services organizations, particularly revenue attribution, financial reporting, resource planning, and utilization analytics.
- Experience operating within SOC 2, ISO 27001, or comparable security and compliance frameworks.
- Current Microsoft data engineering or Power BI certification.
