Engineering Stream
Engineering Stream
Are you focused on market data engineering? Roll your sleeves up and join the market data engineering workshop. Cochaired by Goldman Sachs and JP Morgan.
The Engineering Stream is open to all member categories. Join to hear insights from fellow engineering specialists and share your thoughts and experiences on Monday and Tuesday afternoon. This workshop will enable you to touch, see, and use market data in the cloud with experts on hand to facilitate.
Note: You will need to bring your own laptop to participate
To attend, please contact David Anderson. All attendees also need to register as general WFIC Delegates.
The Market Data Engineering Initiative is chaired by:
John Chappell, Global Head of Market Data, Goldman Sachs
Alistair Kerr, Global Head of Market Data Services, JP Morgan
The Engineering Stream is facilitated by:
Simon Clerck, Global Head of Market Data Engineering, Goldman Sachs
Nigel Phelan, Chief Architect for the Market Data Team, JP Morgan
| Time | Monday 17th October | ||
|---|---|---|---|
| 1:45 p.m. - 1:55 p.m. |
Day 1 Introduction Speaker: Simon Clerck – Global Head of Market Data Engineering – Goldman Sachs |
||
| 1:55 p.m. - 2:35 p.m. |
LSEG: After nearly a decade of being cloud-first, what have we learned?
Hear directly from LSEG engineers on the transformative capabilities and hard learnt lessons of building market data services in the cloud. Having been cloud-first since 2015, LSEG will highlight the challenges of migrating existing services, such as petabytes of Tick History data, as well as the engineering choices possible for born-in-the-cloud capabilities, like MayStreet. Speakers : Oli Bage, Head of Architecture, Data & Analytics - LSEG Philip Hanson, Lead Software Engineer – LSEG Paul Murphy, Head of Solution Architecture - LSEG |
||
| 2:35 p.m. - 3:15 p.m. |
CME: Real-time data in the cloud
In this session, CME Group will discuss its journey of delivering all its real time futures and options data anywhere in the world via the cloud and the challenges that came with it including throughput, ordering and entitlements. Speaker: Tim Wheeler, Director- Product Management- Data Platforms – CME Group |
||
| 3:15 p.m.- 5:15 p.m. |
Lab Class – Market Data in the Cloud
A hands-on opportunity for all attendees to get to know, experiment with and learn about the possibilities of Market Data in the Cloud. Google Cloud have kindly drawn up the exercises and created a ‘cloud lab’ environment for all attendees to use free of charge. This lab will demonstrate how to use Google Cloud to ingest and transform data from a variety of market data vendors using a native binary/encoded form. You will be able to experiment with transcoding the binary/encoded data and creating a stream using Pub/Sub. Dataflow can then also be used to process streaming data from historical datasets, and then store the results in BigQuery for analysis. The exercises will use data from more than one well-known data provider. To participate in the Lab Class, it is required that you bring your own laptop. |
||
| Time | Tuesday 18th October | ||
|---|---|---|---|
| 1:30 p.m. - 1:40 p.m. |
Day 2 Introduction Speaker: Simon Clerck – Global Head of Market Data Engineering – Goldman Sachs |
||
| 1:40 p.m. - 3:00 p.m. |
DRM & ODRL - Technical Deep Dive (Goldman Sachs, JP Morgan, Deontic, AWS)
Session 1 - 20 mins: The AWS Data Exchange and Open Source Legend technologies improving data ingestion, curation & analysis to derive insights faster and more reliably Delivering insights to institutional clients is core to Goldman Sachs’ business, but increasingly complex demands require additional resources and permissions from third-party data and cloud-services providers. In this session, learn how Goldman Sachs combines services to discover, analyse and share curated internal and third-party data in minutes. This demo will illustrate how London Stock Exchange Group’s (LSEG) ESG data delivered to Goldman Sachs through AWS Data Exchange can be queried, combined and used in the same way as first-party data by utilizing Amazon Redshift. Speakers:Emily Menzies, VP Market Data Manager – Goldman Sachs Gerrard Cowburn - Solutions Architect - Amazon Web Services Session 2 - 30 mins: Automation of / confidence in data usage through DRM automation Leveraging the Open Digital Rights Language (ODRL) and the rights management services provided by Deontic Data to demonstrate how the ESG data licensed by Goldman Sachs from LSEG and sourced from ADX can be shared with the bank's clients to support investment decisions while transparently enforcing LSEG's contractual and commercial imperatives. Speakers: Caspar MacRae, Executive Director/Developer – Goldman Sachs Session 3 - 30 mins: Experiments in using an ODRL policy store to control access to cloud data JP Morgan Chase have been working with technology partners to test the practicality of using ODRL representations of market data contracts as the primary source for data access controls in cloud native data stores. This short presentation discusses some initial observations and findings. Speaker:Nigel Phelan – Chief Architect – JP Morgan |
||
| 3:00 p.m. - 3:30 p.m. | Coffee Break | ||
| 3:30 p.m.- 5:00 p.m. |
TechNet Sessions
FactSet (30 mins): In 2019 FactSet started the initiative of putting their Real time ticker plant for 250+ venues and distribution architecture into the cloud. In the presentation the Lead Architect for this initiative will present his approach and learnings from this multi-year project. The following areas will be discussed:
Deutsche Boerse (15 mins): What 15 Years in HFT Has Taught Me about Market Data Some stories behind what made me learn the following truths: (1) Every assumption you make about market data is wrong. (2) Always instrument for failure. (3) Fail early, fail hard. (4) Never ignore rare oddities. (5) Just because something has been in production for a long time does not mean it is correct. (6) I hate Jupyter notebooks. Stefan Schlamp , Head of Content Development & Management – Deutsche Boerse Solace (15 mins): Transport Layer in the cloud The provision of real-time, streaming, market data in the cloud presents many challenges for the provider. Not least of these is choosing an effective and efficient transport capability that scales to support the service, provides acceptable latency and the correct message order guarantees. Adding the need to be able to satisfy a hybrid and multi-cloud capability further complicates the requirements. In this brief session we'll explore the requirements for a transport layer in more depth and look at some of the options that are available for streaming market data in the cloud. Speaker: Mat Hobbis - VP Global Accounts & Services Architecture - Solace Systems Push Technology (15 mins): Market Data in the cloud: Maintaining stability in volatile times As Market Data volumes continue to grow with increasing unpredictability, Market Data platforms must adapt to handle data spikes. In this talk, we will discuss how Diffusion assists with observability of data volumes, dynamically scaling up/down when needed to handle volumes, promoting stability whilst reducing data costs. Speaker: Dave Carson, Head of Field Architecture and Services – Push Technology Bloomberg (15 mins) Measuring Performance in the Cloud: Lessons Learned Extending Market Data Latency Monitoring to the Public Cloud We will share a brief overview of Bloomberg’s internal latency monitoring infrastructure for its real-time market data system, followed by a discussion about several examples of cloud-specific challenges and how we addressed them. Speaker: Paul Callaway - Head of Real-Time Systems Architecture - Bloomberg |
||
