Connecting IT Professionals to the Future
Back to jobs list

Sr Big Data Engineer, AWS RedShift

Design, develop and maintain large-scale real-time event and log data processes in AWS/RedShift environment.  Design and develop scalable data stores with sub-second query latency on highly multi-dimensional data.   

This team is creating an enterprise level, Big Data, columnar database, columnar distributed data warehouse processing systems product, using machine learning.  This team is expanding due to growth.  You can grow your skills in this team and company in big data and columnar databases, and Redshift.

Redshift is ideal, but do not want to filter candidates out because of no Redshift experience. 
People with in-depth Vertica, SQL server, etc, work well also.

Excellent SQL skills with some columnar database or some NOSQL database or some RedShift experience. 

Do your best to create systems with high availability, scalability and reliability of data processing components.   Perform capacity planning and cost estimates of proposed solutions.  Design key performance indicators for monitoring systems to enable the DevOps team to prepare for monitoring of systems.  Design with DevOps in mind, to simplify DevOps job. Bring in new data sets.  Expose data through API’s.  Lead the design meetings and code review sessions, and mentor junior programmers or team members.  This requires a hard core SQL resource, and team player.

Here are some specifics on the skillset sought: 
SQL (any database) – advanced SQL Required.

  1. Windowing Aggregate functions: there should be extensive knowledge of how they are used and how they work internally (query planning and execution).
  2. Common Table Expressions (CTE): usage and complete understanding of how they help.  Recursive CTE is a bonus, but not a requirement.
  3. Correlated Subqueries: there should be extensive knowledge of how and when to use those.
  4. Data Modeling: both logical and physical, especially in columnar databases.  Entity Relationship modeling, normalization (3NF) to third normal form, de-normalization and columnar compression encodings are extremely important.
  5. Query Plan reading, comprehension and optimizations, especially in distributed databases.  Understanding of data moves, data sharding, skews, etc. are essential.

Here is the career direction, and their environment:
any experience with Enterprise Data Warehouses or Distributed Data Warehouses, in example:  ExaData, GreenPlum, TerraData -  focus on Data Warehousing - Enterprise Data Warehousing - how the entire enterprise data warehouse works with columnar databases, meta data extraction, ETL, a data model design keeping devops’ job as simplified and efficient as possible.  A simple Redshift DW design. They process billions of records each day.

Massively parallel computer architecture, Massively Parallel Processing (MPP)
Cloud Computing, Machine Learning, A.I.

data API’s

Big Data system architecture
database structure
data warehouse stores and retrieval

message brokers, message queueing technologies
AWS (amazon web services)

AWS RedShift, Amazon RedShift

Apache Spark 2.0 architecture

distributed systems architecture

Apache Kafka
Apache Hadoop

Apache Cassandra
Apache Storm

Employment type:  Regular Full-Time, salary + bonus

Salary Range:  $140,000 to 185,000 per year

                           Plus annual bonus up to 20-25% of salary

Benefits:  life, health, dental, vision, 401k matching, ADD, short-term disability, paid vacation, paid sick time, bonus up to 20-25% of annual salary

#1 location:  Santa Ana, California
#2 location:  Palo Alto, California

Both offices work.  Either one is fine. 

Please apply with resume, MS Word format preferred.

You may apply with your Linked IN profile and/or your resume from this Online Job Posting URL:

Please call 817-424-1411 to inquire or to follow up on your resume submittal. 
If you need a toll-free number, please use:  888-416-8999  extension  3001.

Immigration:  US citizens and those authorized to work

To email your resume or to send an email inquiry, click here