New Update to Historical Market Data APIs

Developer API
1

Hey folks! :waving_hand:

A couple of weeks ago, we had a few users who started requesting data at a abnormal rates. 6 years data, 1min timeframe, and the same fetch for a more new instruments in the next second.

This was not new. Weā€™ve seen a few newbie users whoā€™re just learning and implying do that. But this time it was a group of users who all got active and sent requests at the same time.

Since we pass on the TBT saved data over these APIs directly from our main Database (ofcourse thereā€™s a wrapper on top, but still the same source that powers charts on our platforms), the performance of our platforms was disturbed. We had to take a swift action to stop this and put a hard rate limit for our platforms to get back to normal.

Weā€™ve made a new wrapper on top now that would filter such requests and only pass the ones which are in limits. The limits are as follows:

1. For 1 to 15 min : 1 month of data in a single request
2. For 16 to 300 min: 3 months (1 quarter) data in a single request

Iā€™m sorry for the silence when your systems had a disruption, but I only wanted to talk once we had a solution in place and a quick patch that could serve more than 5 days of data.

Weā€™ll be updating these limits in the documentation too.

Thank you,

Best,
Mohit
Team Upstox

4 Likes
2

@MohitGolecha

Thanks for the updates. Helps to get some clarity, few questions

  • Is it confirmed the limits are here to stay and not going to degrade further?
  • Can you consider changing the limit for 15 min candle to 1 qtr at the very least, that should be reasonable load still on your systems
  • Can you have some sort of official status page for apis, so that in case of disruptions we have a clear communication channel and not devs giving random updates with no clarification?
  • Please consider building a ā€œtrustedā€ users system, which will allow some of us with valid use cases to get extended data. I understand it might take some time to build it, but shouldnā€™t be too complex.

Thanks

1 Like
3

Hi @Ankit_Agarwal

  1. Weā€™re trying to balance offering APIs for free for everyone. Weā€™ve had instances where one client uses gigabytes of data for historical data which degrades the quality of performance for everyone. Hopefully 1 month should suffice - so far it looks good. Will keep you posted

  2. Can think about it - we donā€™t have any rate limits for historical data so approaching it cautiously to balance performance/cost vs giving as much as possible to clients

  3. We have put up Upstox Status Website for any outages on the Upstox ecosystem.

  4. Noted! For now, we have introduced Plus Plan (with no added subscription fee) to give enhanced API capabilities.

Thanks for the feedback - please keep giving us more feedback as we continue to improve API for all :slight_smile:

5 Likes