Ovo Energy
Tech Lead & Senior Software/Data Engineer, Ovo Energy Aug 2022 – Present
Key technologies: Python, Airflow, SQL (Postgres & BigQuery), Terraform, Google Cloud Platform (GCP), CI/CD with GitHub actions, a little Kafka & Kubernetes (GKE)
I am currently the tech lead of a wonderfully diverse and talented team of data engineers specialising in the collections domain. My role is a mixture of planning the strategic direction for our technical architecture; logical analysis to ensure we are building the right thing; and detailed problem solving with code & collaboration. A particular focus is coaching colleagues whilst nurturing a strong sense of shared responsibility for ensuring data is accurate & timely so that our customers are treated fairly.
Current Health (a Best Buy Health company)
Software/Data Engineer, Current Health Sep 2020 – Aug 2022
Key technologies: Terraform, SQL (Postgres & Redshift), Kinesis, Lambda, AWS IoT, AWS IAM, Databricks
Owned the creation of technical infrastructure using Terraform of our near real-time streaming of data from our second-generation wearable device to consume events from AWS IoT and coordinate AWS Lambda functions with AWS Kinesis streams and SQS queues. Key contributor to creating a Redshift cluster using federated query to enable clients to access data on incredibly tight deadline. Often maintain the infrastructure behind our Databricks installation. An important part of my role is collaborating with colleagues via pair programming and creating resources (documentation, diagrams etc.) to share knowledge across our remote team.
FanDuel
Software/Data Engineer (Python), FanDuel Jun 2018 – Aug 2020
Key technologies: Python, EMR, S3/Spectrum data lake, Redshift, SQL, Postgres, MySQL, Git, Linux/Bash, Terraform
Ingestion of a large and complex dataset from a variety of sources (Postgres and AWS Aurora/MySQL databases; S3 and external APIs), building a data lake from the ground up with data in S3/Spectrum and Amazon Redshift and Airflow, including scripting and applying database migrations with Alembic and data both via batch and streaming sources. Accustomed to data in a variety of formats including Avro and Parquet. Contributed to infrastructure for a new data lake using Terraform; and AWS Glue (Apache Spark) jobs to pick up and transform data from our external vendors. We maintained Apache Spark, Hadoop and Hive on an Elastic Map Reduce (EMR) cluster, which sometimes involved troubleshooting to diagnose the root cause of a failure. Wrote and delivered a Python training course to a diverse audience of c50 people.
JP Morgan Chase & Co
Senior Associate Applications Developer and Scrum Master
Oct 2016 – Jun 2018
Key technologies: Python (including some Django), Neo4j, Git, Linux/Bash
Wood Mackenzie
Senior Data Engineer, Wood Mackenzie Data Strategy & Mgmt / Data Science
Jan 2015 – Oct 2016
Key technologies: Python, Git, SQL Server, Neo4j, Cypher, MS Excel, MS Access
My focus was on developing Python scripts to integrate with and enhance existing applications. I developed an oil price model using Python for data processing and linear programming with a front-end in MS Excel. This was the centrepiece in a commercially successful consulting project for a valued client.
Throughout my time at Wood Mackenzie I was often responsible for taking the lead on the data aspect of complex projects. My most satisfying moments were when I took a time-consuming manual process and automated it to produce something that was slick and intuitive. I launched data governance at Wood Mackenzie by performing a survey of data provenance, compiling the results into a Neo4j graph database and making the results available for my colleagues to access, making metadata easy to find and saves countless days on impact analysis by building the picture of how data flowed through the company.
Senior Data Analyst, Wood Mackenzie Energy Markets Team
Apr 2010 – Jan 2015
Key technologies: SQL Server, SQL Server Integration Services (SSIS), MS Excel, MS Access, Tibco Spotfire, VBA
I built and improved numerous MS Excel & MS Access processes to increase robustness, usability, and reliability. Developed a SQL server database of power plant data. My work made it possible for the team to transform a spreadsheet-driven approach into a global dataset which was subsequently launched to clients. I used SQL Server Integration Services (SSIS) to create an efficient transfer of data from the MS Excel front-end, which achieved quick, consistent and reliable performance for all colleagues. This was a feat which had never before been managed in the business, and I achieved it despite never having used SQL Server or SSIS previously.
I assumed ownership of the complex topic of units and conversion factors, which are vital in the energy industry. This involved researching and calculating every single energy conversion from scratch. I released my findings in a publication to every single client, a privilege only granted to a select few analysts. Building on my achievement, I developed an Excel & VBA tool which was rolled out to all employees, and my work made it possible to improve consistency in this vital business process.
Other noteworthy achievements:
- Rebuilt the web-based Wood Mackenzie Energy Markets data tool using Tibco Spotfire. This directly contributed to an increase in client usage of 240% in 2013.
- In 2014 I was promoted to a line manager.
Scottish & Newcastle Pub Company (SNPC)
Finance Analyst
Oct 2007 – Apr 2010
Key technologies: MS Excel, MS Access, VBA
The focus of my role was in redeveloping processes to increase team efficiency, primarily using MS Excel and VBA. I used MS Access, despite having no prior experience with this application, to rebuild a reporting system to monitor capital & acquisition expenditure returns for Leadership Team reporting. Due to my extensive knowledge of Microsoft applications I delivered a training course on MS Excel to internal staff.