Senior Big Data Engineer (Scala / Spark)

Sydney, New South Wales, Australia | Engineering | Full-time | Partially remote


Quantium are continuing to hire with all interviewing and on-boarding done virtually due to COVID-19. We look forward to meeting you in person when our offices re-open and it is safe to do so.

The company

Founded in 2002, Quantium combines the best of human and artificial intelligence to power possibilities for individuals, organisations and society. Our solutions make sense of what has happened and what will, could or should be done to re-shape industries and societies around the needs of the people they serve.

The product: Q.refinery

Working with Scala, Spark and the rest of the Quantium tech ecosystem, you’ll be building applications to work with large, complex transactional data sets that provide meaningful insights to some of the world’s largest banks. Q.Refinery has been rolled out in multiple geographies, running complex algorithms at scale (tens of billions of records!) and with an ambitious roadmap for expanding the platform we are looking for talented Senior Big Data Engineers to join the team.

We appreciate Scala is not the most common language in the world and have developed a structured L&D program to accelerate your learning and bring you up to speed with all things Scala and distributed computing. If you have experience with a JVM based language and are looking to upskill / cross-skill, Quantium offers a fantastic opportunity to do so whilst working on a variety of interesting Spark and Scala applications across the platform.

Q.Refinery is NOT a simple Spark ETL, it requires critically applied Software Engineering principles to scale. Nor are we functional purists, we write pragmatic functional code to solve unique complex problems that require out of the box thinking. If you enjoy solving challenging engineering problems first and foremost then this could be the role for you.

The role

You will be responsible for developing readable, maintainable and efficient code to realise user stories that deliver the product road-map. You will be accountable for coordinating engineers and other stakeholders in breaking down large epics into collections of smaller stories / tasks and implementing the overall functionality.

Through active participation in Scrum ceremonies, you will contribute to the product road-map and collaborate with other Engineers and Analysts to assist in defining the technical direction for the product. You will also work with support teams to ensure the product's smooth operation.

Working as part of a cohesive and collaborative cross-functional team, you’ll be rubbing elbows (because social distancing of course!) with product managers, UX/UI designers, analysts, testers and fellow engineers. You’ll make a positive contribution as an expert in your field who is excited by the idea of working across our stack and growing your "T" shape. 

Whilst writing well-tested code day to day may be a given, here are a few other ways you’ll contribute at Quantium:

  • influencing what we build and how we build it. We have a flat structure that means YOU will have the opportunity to own solutions,
  • building up the skill of the team, and in turn picking up other areas of our stack,
  • participating in our agile ceremonies,
  • performing code reviews,
  • wrangling CI, CD, and local development to "sharpen the saw",
  • contributing to core services and libraries,
  • supporting our customer success team, and keeping our production environment humming

Our new team member

Here are a few things we are looking for you to bring to the team:

  • You'll have experience developing applications at scale using a JVM based language
  • Have had some exposure to big data tech (Kafka, Spark, Cassandra, Cosmos DB, Flink, ElasticSearch, etc)
  • Be able to discuss and explain complicated technical systems that you are working on
  • You value delivering high-quality, peer-reviewed, well-tested code
  • Awareness of DevOps functions and appetite to contribute to CI / CD pipelines
  • Experience working with source control tools (GIT preferred) with good understanding of branching / merging strategies
  • Comfortable working in a fast moving, agile development environment
  • Excellent problem solving / analytical skills
  • Good written / verbal communication skills
  • Commercially aware, with the ability to work with a diverse range of stakeholders
  • When it comes to architecture and solution design discussions, you can advocate your ideas and critique other's constructively.
  • Must have a passion for mentoring others, as well as setting goals for your own learning and development – we’re all about growth

More about Quantium

We are passionate about creating opportunities for ourselves and each other. You will have access to relevant, meaningful, and actionable learning. We are growing our technical, professional, and leadership capabilities through our robust internal learning curriculum, team knowledge-sharing sessions, and our mentoring program.

We also believe there is more to a successful career than just hard work and recognise the need to balance personal and family commitments with work. We enjoy weekly group fitness sessions (being done virtually at the moment), team sports (we are looking forward to these starting again), and well-being resources.

Quantium are an equal opportunity employer and values diversity. We are a multicultural team of various beliefs, sexual orientations, and gender identities. We celebrate our diversity of race, religion, colour, national origin, gender, sexual orientation, age, marital status, and disability status.