Distributed Database Engineer

Distributed Database Engineer

Washington DC,District of Columbia,United States

Software and Services

We're in search of exceptional software engineers who possess extensive expertise in building large-scale distributed databases, tackling challenges such as SQL optimization, vectorized engine execution, SIMD vectorization, and distributed query execution for Apache Spark. In this role, you will be at the forefront of pushing the boundaries in building a native SQL engine accelerator for Spark, operating at Apple's scale for big data analytics, and optimizing every aspect of performance. We are particularly interested in engineers who have a deep understanding of systems like Spark, Trino, Presto, Arrow, or other relevant frameworks. We are also seeking individuals who are enthusiastic about the opportunity to collaborate closely with internal Apple teams as well as external communities.

Description

Our teams are tasked with providing a Data Platform as a service, utilizing technologies such as Spark, Trino, Iceberg, Druid, and Flink. This platform is made available to multiple internal departments within Apple and serves as the fundamental infrastructure for a wide range of data engineering and data science applications. Improving the efficiency of Apache Spark would have significant benefits. It would enable faster delivery of data insights, leading to better business outcomes. Moreover, it would result in substantial cost reductions in compute expenses, considering the scale of Spark usage at Apple. In our team, there are multiple individuals who play significant roles as project management committee (PMC) members and committers for projects like Spark, Flink, Arrow, Parquet, Iceberg, and other related initiatives that we actively utilize. We have been dedicated to enhancing these open-source projects through our upstream contributions. Notably, several team members received recognition through SIGMOD System Awards for their contributions to Spark in 2022 and Flink in 2023. In order to enhance Spark's performance, we are developing a native engine for Apache Spark using Apache Arrow and Arrow DataFusion as the foundation. This engine is designed to replace Apache Spark's current execution engine, which is based on the JVM, with a new, vectorized engine written in a low-level system language. This implementation is already in use at Apple and has demonstrated significant enhancements in query performance and more efficient resource utilization. You'll get the chance to collaborate with them in enhancing the native SQL engine and related technologies to a higher standard. This will involve working closely with both open-source communities and internal teams. Here you'll find that we: - Have a peaceful work environment where you can choose when to socialize and when to focus. - Provide technical guidance and mentorship as needed - Have an encouraging and growth-focused workplace culture. - Encourage you to use accrued vacation time and maintain a healthy work-life balance. - Offer excellent benefits!

Minimum Qualifications

• A proven history of success or a showcased talent as an engineer with experience or academic research background in distributed database systems.

• Experience on query compilation & optimization, vectorized engine execution, SIMD, distributed query execution, database design, and/or performance optimization.

• Proficient in system languages such as C, C++, Rust, or Swift.

Key Qualifications

Preferred Qualifications

• A contributor, committer, or PMC member for Apache Spark, Apache Arrow, or related database technologies

Education & Experience

Additional Requirements

Pay & Benefits

• At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $124,600 and $230,600, and your base pay will depend on your skills, qualifications, experience, and location.Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation.Learn more (~~~) about Apple Benefits.Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

• Apple is an equal opportunity employer that is committed to inclusion and diversity. We take affirmative action to ensure equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics.Learn more about your EEO rights as an applicant. (~~~)

Apple Footer

Apple is an equal opportunity employer that is committed to inclusion and diversity. We take affirmative action to ensure equal opportunity for all applicants without regard to race,color,religion,sex,sexual orientation,gender identity,national origin,disability,Veteran status,or other legally protected characteristics. Learn more about your EEO rights as an applicant (Opens in a new window) .

Apple will not discriminate or retaliate against applicants who inquire about,disclose,or discuss their compensation or that of other applicants. United States Department of Labor. Learn more (Opens in a new window) .

Apple will consider for employment all qualified applicants with criminal histories in a manner consistent with applicable law. If you're applying for a position in San Francisco,review the San Francisco Fair Chance Ordinance guidelines (opens in a new window) applicable in your area.

Apple participates in the E-Verify program in certain locations as required by law. Learn more about the E-Verify program (Opens in a new window) .

Apple is committed to working with and providing reasonable accommodation to applicants with physical and mental disabilities. Apple is a drug-free workplace. Reasonable Accommodation and Drug Free Workplace policy Learn more (Opens in a new window) .