Software Development/Data Engineer

This Opportunity

WSP is currently initiating a search for a Software Development/Data Engineer to join our Bridge Inspection Group at Penn Plaza, New York City, NY office. Be involved in projects with our Bridge Inspection Team and be a part of a growing organization that meets our client's objectives and solves their challenges

Your Impact

This role is responsible for the design, deployment, development, and maintenance of data solutions.

• Participate in a variety of data-related projects and work closely with Business users, and other stakeholders to gather requirements and build data pipelines that meet the organizations' needs.

• Deploy and support Data platforms that process and store data from structural engineering, bridge design and bridge inspection in required format.

• Contribute to software development methods, tools, and techniques and apply agreed standards and tools to achieve well-engineered outcomes.

• Design, develop, and implement data solutions to meet business requirements and data ingestion needs, facilitating accurate and timely data availability for analysis

• Extract, load, and Transform (ELT/ETL) data from various sources, including on-premises and cloud-based systems, APIs, databases, and files.

• Develop APIs as needed to transform data in required format from various sources of bridge design and inspection.

• Write well-designed, efficient code that adheres to security standards.

• Monitor and troubleshoot data pipelines to identify and resolve issues promptly to minimize disruptions in data processing for on-premises and cloud environments.

• Implement data quality checks and validation processes to ensure accuracy and completeness of data.

• Write complex queries and scripts to efficiently manipulate, transform, and process raw data from bridge inspection.

• Create and execute data validation processes to ensure the reliability and consistency of incoming data.

• Build processes to monitor data quality.

• Continuously optimize data pipelines for performance, scalability, and reliability

• Create and maintain technical documentation.

• Contribute to building and maintaining data catalog and lineage.

• Design and develop CI/CD processes that ensure high availability and agility.

• Develop cloud data services provisioning automation with integrated capabilities of IAM, network, security policies as code, and observability.

• Build tools and services to support data discovery, lineage, resiliency, and privacy compliance across the data platform.

• Stay updated with the latest trends and best practices in data engineering, cloud computing, and Azure services to suggest innovative solutions to continually improve the organization's data intelligence capabilities.

• Monitor and report on supplier performance, customer satisfaction, adherence to security requirements and market intelligence.

• Remain current in latest discipline-specific techniques and practices.

• Collaborate with professionals from a variety of disciplines to provide future ready solutions for clients.

• Exercise responsible and ethical decision-making regarding company funds, resources and conduct, and adhere to WSP's Code of Conduct and related policies and procedures.

• Perform additional responsibilities as required by business needs..

Who You Are

Required Qualifications

• Bachelor's Degree or equivalent in IT, Computer Science, Engineering, or related fields.

• 3+ years of relevant post education experience.

• Strong knowledge of Big Data architectures, large data warehouses, and Data Lake solutions.

• Experience designing and implementing Modern Data and Analytics solutions including Lakehouse architecture and medallion architecture.

• Proficient in cloud services including Azure Databricks.

• Azure Synapse Analytics, Azure Data Factory, Azure Data Lake Store, Microsoft Purview, and Power BI.

• Experience with major database platforms including Oracle, SQL Server as well as Cloud databases and NOSQL Databases.

• Strong understanding of data engineering concepts, ELT/ETL principles, and data modeling.

• Experience with data integration techniques for both structured and unstructured data.

• Solid programming skills in languages such as Python, Pyspark, and SQL.

• Experience with Airflow.

• Experience in DevOps, Git Repos, and CI/CD pipelines for code deployment.

• Experience deploying and administering cloud-based data solutions using infrastructure-as-code and infra-automation tools like Terraform, Ansible, etc.

• Strong knowledge of Microsoft Azure Cloud.

• Experience with Jira, Confluence.

• Proficiency in at least one software engineering methodology, including but not limited to Agile, Scrum, DevOps, Extreme Programming (XP), Kanban, Lean, and Rapid Application Development (RAD).

• Proven track record of upholding workplace safety and ability to abide by WSP's health, safety and drug/alcohol and harassment policies.

• Must be flexible to a variety of schedules to meet business needs and able to prioritize responsibilities and quickly adapt to change in a pressure work environment.

Preferred Qualifications

• Masters Degree is a plus.

• Prior experience in handling and processing data for structural design and bridge design and inspection is preferred.

• Experience applying structured validation and testing methods, including but not limited to Unit Testing, Integration Testing, System Testing, Acceptance Testing, and Regression Testing.

WSP Benefits:

WSP provides a comprehensive suite of benefits focused on a providing health and financial stability throughout the employee's career. These benefits include coverage related to medical, dental, vision, disability, and life; retirement savings; paid sick leave; paid vacation (or other personal time); paid parental leave; and paid time off for purposes of bereavement, voting, and/or attendance at naturalization proceedings.

Compensation

Expected Salary (all locations): $73,100.00 - 125,400.00

WSP USA is providing the compensation range that the company in good faith believes it might pay and offer for this position, based on the successful applicant's education, experience, knowledge, skills, abilities in addition to internal equity and specific geographic location. WSP USA reserves the right to ultimately pay more or less than the posted range and offer additional benefits and other compensation, depending on circumstances not related to an applicant's sex or other status protected by local, state, and/or federal law.

#LI-PR1