changing Bid_Price from real to float to avoid Bid_Price, Offer_Price type inconsistency
handle (ignore) additional flags TradedOnMEMX and TradedOnMIAX in master file https://www.nyse.com/publicdocs/nyse/data/Daily_TAQ_Client_Spec_v3.3.pdf
handle (ignore) additional flag TradedOnLTSE in master file https://www.nyse.com/publicdocs/nyse/data/Daily_TAQ_Client_Spec_v3.2.pdf
small changes to field load types (mostly just wider) - see before/after versions of nf2, tf2, qf2
support NYSE change to schema as specified in: https://list.theice.com/t/92262/395348/57007/0/
support NYSE change to schema, format and datacentre as specified in: http://www.nyxdata.com/doc/247075
taq.k reads the new data files, but saves data to the old schema, discarding nano-second timestamp precision. It is a drop-in replacement for the previous version for those who don't want to move to the new schema yet.
tq.q reads and exports ALL fields for trade, quote and nbbo
Timestamp precision extended from millisecond to microseconds (additional 3 digits ignored for now). Ignore 3 additional fields participant timestamp, RRN and TRF http://www.nyxdata.com/nysedata/default.aspx?tabid=993&id=2784
Support additional exchange codes (12-15 - ZJKY) in taqmaster
Amend taq.k to discard the first line of taqmaster* when it is just lists the record count
Amend taq.k and tq.q to recognise and ignore the new quote fields:
- SIP-generated Message Identifier
- National BBO LULD Indicator tq.q to recognise and ignore the new NBBO fields:
- Limit-Up/Limit-Down Indicator
- Limit-up/Limit-down NBBO Indicator
- SIP-generated Message Identifier and handle deletion of previous NBBO fields:
- Limit-Up/Limit-Down NBBO (UTP) Indicator
- Limit-Up/Limit-Down NBBO (CQS) Indicator
The format change is scheduled to take effect from 2nd December 2013 see: http://www.nyxdata.com/nysedata/Default.aspx?tabID=993&id=2194
Amended taq.k and tq.q to recognise and ignore the new quote fields:
- Short Sale Restriction (SSR) Indicator
- Limit-Up/Limit-Down BBO (UTP) Indicator
- Limit-Up/Limit-Down BBO (CQS) Indicator
- FINRA ADF MPID Indicator and tq.q to recognise and ignore the new NBBO fields:
- Limit-Up/Limit-Down NBBO (UTP) Indicator
- Limit-Up/Limit-Down NBBO (CQS) Indicator
The format change is scheduled to take effect from 1st February 2013 see: http://www.nyxdata.com/nysedata/default.aspx?tabid=993&id=1771 http://www.nyxdata.com/doc/185107
Amended taq.k and tq.q to recognise (and ignore) the new Quote field RPI (Retail Interest Indicator)
Added tq.q as example of how to load more fields from the FTP files. taq.k is unchanged.
Adjust partitioning used with -par to split at a whole symbol – so any particular symbol will only be in one partition.
Enabled to handle >2billion rows in input file. Use in combination with -par cmd line option and par.txt
Amend the file detection code to pick up the new file names (as well as the old ones) when NYSE changes the filenames for the FTP download on September 17th, 2010.
You are welcome to download and use this code according to the terms of the licence.
Kx Systems recommends you do not link your application to this repository, which would expose your application to various risks:
- This is not a high-availability hosting service
- Updates to the repo may break your application
- Code refactoring might return 404s to your application
Instead, download code and subject it to the version control and regression testing you use for your application.