Live Data Feed from the WebSocket returns only subset of requested data

Developer API
1

Live Data Feed from the WebSocket returns only subset of requested data. If I request data for 100 instruments, then it returns only a subset of instruments. Are you setting any limits for the websockets ?

2

Could you assist with the list of instrument keys here? This issue typically arises when the instrument keys are invalid or associated with an expired contract.

When the instrument keys are invalid in the list, the Websocket will only subscribe to the valid instrument keys and begin providing data for those specific keys.

3

The instrument keys are pulled up from https://assets.upstox.com/market-quote/instruments/exchange/NSE.csv.gz
Since I am not using any other type of instruments except for NSE_EQ stocks, there is no issue because of expired contract. Also, I get the full list the first time i poll it but later it drops to a very small subset.

4

Could you kindly provide your client ID? Additionally, if possible, I would appreciate it if you could compile a list of instruments in a file and attach it here for detailed examination.

5

The data points that I shared hold good for your tests and resolution. I suspect some form of rate limiting is getting triggered. At first, all requested set of instrument data is returned and later all requests for the same set returns less than 20% of the whole set.

6

Okay. We will investigate and get back to you.

7

Hi @Pradeep_Jaiswar , @SimS,

I am facing the same issue, I am subscribing to 2 instruments initially, and for the 1st time I am getting both the instruments in the websocket feed, but after I recieve 1 or 2 feeds, then after that for the 3rd feed, websocket sends either of the 1 instrument.

Initial Subscribe List :- ['NSE_INDEX|Nifty 50', 'NSE_EQ|INE002A01018']
Unsubscribe Symbol :- None

[Websocket Connection Established]

{'feeds': {'NSE_INDEX|Nifty 50': {'ff': {'indexFF': {'ltpc': {'ltp': 21979.6, 'ltt': '1710734908000', 'cp': 22023.35}, 'marketOHLC': {'ohlc': [{'interval': '1d', 'open': 21990.1, 'high': 22045.3, 'low': 21953.7, 'close': 21979.6, 'ts': '1710700200000'}, {'interval': 'I1', 'open': 21961.05, 'high': 21977.45, 'low': 21961.05, 'close': 21971.0, 'ts': '1710734820000'}, {'interval': 'I1', 'open': 21971.1, 'high': 21979.6, 'low': 21967.5, 'close': 21979.6, 'ts': '1710734880000'}, {'interval': 'I30', 'open': 22005.15, 'high': 22045.3, 'low': 21953.7, 'close': 21979.6, 'ts': '1710733500000'}]}, 'yh': 22526.6, 'yl': 16913.75}}}, 'NSE_EQ|INE002A01018': {'ff': {'marketFF': {'ltpc': {'ltp': 2844.95, 'ltt': '1710734908272', 'ltq': '6', 'cp': 2836.45}, 'marketLevel': {'bidAskQuote': [{'bq': 7, 'bp': 2844.2, 'bno': 1, 'aq': 50, 'ap': 2844.95, 'ano': 1}, {'bq': 156, 'bp': 2844.15, 'bno': 4, 'aq': 218, 'ap': 2845.0, 'ano': 14}, {'bq': 102, 'bp': 2844.1, 'bno': 2, 'aq': 2, 'ap': 2845.05, 'ano': 1}, {'bq': 34, 'bp': 2844.05, 'bno': 2, 'aq': 1, 'ap': 2845.1, 'ano': 1}, {'bq': 399, 'bp': 2844.0, 'bno': 5, 'aq': 1, 'ap': 2845.15, 'ano': 1}]}, 'optionGreeks': {}, 'marketOHLC': {'ohlc': [{'interval': '1d', 'open': 2840.0, 'high': 2854.95, 'low': 2836.45, 'close': 2844.95, 'volume': 477370, 'ts': '1710700200000'}, {'interval': 'I1', 'open': 2843.0, 'high': 2844.95, 'low': 2843.0, 'close': 2844.5, 'volume': 5024, 'ts': '1710734820000'}, {'interval': 'I1', 'open': 2844.5, 'high': 2844.95, 'low': 2844.0, 'close': 2844.95, 'volume': 1820, 'ts': '1710734880000'}, {'interval': 'I30', 'open': 2839.0, 'high': 2854.95, 'low': 2836.45, 'close': 2844.95, 'volume': 459306, 'ts': '1710733500000'}]}, 'eFeedDetails': {'atp': 2845.37, 'cp': 2836.45, 'vtt': '477381', 'tbq': 142325.0, 'tsq': 165170.0, 'lc': 2552.8, 'uc': 3120.05, 'yh': 3024.9, 'yl': 2180.0, 'fp': 2844.95, 'fv': 5}}}}}}
{'type': 'live_feed', 'feeds': {'NSE_EQ|INE002A01018': {'ff': {'marketFF': {'ltpc': {'ltp': 2844.95, 'ltt': '1710734908486', 'ltq': '11', 'cp': 2836.45}, 'marketLevel': {'bidAskQuote': [{'bq': 11, 'bp': 2844.4, 'aq': 39, 'ap': 2844.95}, {'bq': 76, 'bp': 2844.35, 'aq': 218, 'ap': 2845.0}, {'bq': 56, 'bp': 2844.3, 'aq': 2, 'ap': 2845.05}, {'bq': 2, 'bp': 2844.25, 'aq': 1, 'ap': 2845.1}, {'bq': 21, 'bp': 2844.2, 'aq': 1, 'ap': 2845.15}]}, 'optionGreeks': {}, 'marketOHLC': {'ohlc': [{'interval': '1d', 'open': 2840.0, 'high': 2854.95, 'low': 2836.45, 'close': 2844.95, 'volume': 477381, 'ts': '1710700200000'}, {'interval': 'I1', 'open': 2843.0, 'high': 2844.95, 'low': 2843.0, 'close': 2844.5, 'volume': 5024, 'ts': '1710734820000'}, {'interval': 'I1', 'open': 2844.5, 'high': 2844.95, 'low': 2844.0, 'close': 2844.95, 'volume': 1831, 'ts': '1710734880000'}, {'interval': 'I30', 'open': 2839.0, 'high': 2854.95, 'low': 2836.45, 'close': 2844.95, 'volume': 459317, 'ts': '1710733500000'}]}, 'eFeedDetails': {'atp': 2845.37, 'cp': 2836.45, 'vtt': '477392', 'tbq': 141967.0, 'tsq': 166094.0, 'lc': 2552.8, 'uc': 3120.05, 'yh': 3024.9, 'yl': 2180.0}}}}}}
==================================================[Error Occurred]==================================================

Traceback (most recent call last):
  File "C:\Projects\Upstox_v2\UnitTester.py", line 1302, in prev_2_Day_And_Live_Data_Merge_PROD
    first_instance_of_I1 = data_dict["feeds"][str(preferredIndex)]["ff"]["indexFF"]["marketOHLC"]["ohlc"][1]
KeyError: 'NSE_INDEX|Nifty 50'

An error occurred: 'NSE_INDEX|Nifty 50'

Also...
Initial Subscribe List :- ['NSE_INDEX|Nifty 50', 'NSE_EQ|INE002A01018']
Unsubscribe Symbol :- None

[Websocket Connection Established]

{'feeds': {'NSE_INDEX|Nifty 50': {'ff': {'indexFF': {'ltpc': {'ltp': 21978.75, 'ltt': '1710734909000', 'cp': 22023.35}, 'marketOHLC': {'ohlc': [{'interval': '1d', 'open': 21990.1, 'high': 22045.3, 'low': 21953.7, 'close': 21978.75, 'ts': '1710700200000'}, {'interval': 'I1', 'open': 21961.05, 'high': 21977.45, 'low': 21961.05, 'close': 21971.0, 'ts': '1710734820000'}, {'interval': 'I1', 'open': 21971.1, 'high': 21979.6, 'low': 21967.5, 'close': 21978.75, 'ts': '1710734880000'}, {'interval': 'I30', 'open': 22005.15, 'high': 22045.3, 'low': 21953.7, 'close': 21978.75, 'ts': '1710733500000'}]}, 'yh': 22526.6, 'yl': 16913.75}}}, 'NSE_EQ|INE002A01018': {'ff': {'marketFF': {'ltpc': {'ltp': 2844.3, 'ltt': '1710734908655', 'ltq': '5', 'cp': 2836.45}, 'marketLevel': {'bidAskQuote': [{'bq': 1, 'bp': 2844.4, 'bno': 1, 'aq': 28, 'ap': 2844.95, 'ano': 1}, {'bq': 2, 'bp': 2844.25, 'bno': 1, 'aq': 218, 'ap': 2845.0, 'ano': 14}, {'bq': 80, 'bp': 2844.15, 'bno': 1, 'aq': 2, 'ap': 2845.05, 'ano': 1}, {'bq': 5, 'bp': 2844.1, 'bno': 1, 'aq': 1, 'ap': 2845.1, 'ano': 1}, {'bq': 19, 'bp': 2844.0, 'bno': 3, 'aq': 1, 'ap': 2845.15, 'ano': 1}]}, 'optionGreeks': {}, 'marketOHLC': {'ohlc': [{'interval': '1d', 'open': 2840.0, 'high': 2854.95, 'low': 2836.45, 'close': 2844.3, 'volume': 477697, 'ts': '1710700200000'}, {'interval': 'I1', 'open': 2843.0, 'high': 2844.95, 'low': 2843.0, 'close': 2844.5, 'volume': 5024, 'ts': '1710734820000'}, {'interval': 'I1', 'open': 2844.5, 'high': 2844.95, 'low': 2844.0, 'close': 2844.3, 'volume': 2136, 'ts': '1710734880000'}, {'interval': 'I30', 'open': 2839.0, 'high': 2854.95, 'low': 2836.45, 'close': 2844.3, 'volume': 459622, 'ts': '1710733500000'}]}, 'eFeedDetails': {'atp': 2845.37, 'cp': 2836.45, 'vtt': '477697', 'tbq': 140796.0, 'tsq': 166337.0, 'lc': 2552.8, 'uc': 3120.05, 'yh': 3024.9, 'yl': 2180.0, 'fp': 2844.3, 'fv': 5}}}}}}
{'type': 'live_feed', 'feeds': {'NSE_INDEX|Nifty 50': {'ff': {'indexFF': {'ltpc': {'ltp': 21978.3, 'ltt': '1710734910000', 'cp': 22023.35}, 'marketOHLC': {'ohlc': [{'interval': '1d', 'open': 21990.1, 'high': 22045.3, 'low': 21953.7, 'close': 21978.3, 'ts': '1710700200000'}, {'interval': 'I1', 'open': 21961.05, 'high': 21977.45, 'low': 21961.05, 'close': 21971.0, 'ts': '1710734820000'}, {'interval': 'I1', 'open': 21971.1, 'high': 21979.6, 'low': 21967.5, 'close': 21978.3, 'ts': '1710734880000'}, {'interval': 'I30', 'open': 22005.15, 'high': 22045.3, 'low': 21953.7, 'close': 21978.3, 'ts': '1710733500000'}]}, 'yh': 22526.6, 'yl': 16913.75}}}}}
==================================================[Error Occurred]==================================================

Traceback (most recent call last):
  File "C:\Projects\Upstox_v2\UnitTester.py", line 1300, in prev_2_Day_And_Live_Data_Merge_PROD
    await get_BestSeller_Quote(data_dict=data_dict)
  File "C:\Projects\Upstox_v2\UnitTester.py", line 693, in get_BestSeller_Quote
    bestSeller = data_dict["feeds"][str(instrumentKey)]["ff"]["marketFF"]["marketLevel"]["bidAskQuote"][0]["ap"]
KeyError: 'NSE_EQ|INE002A01018'

An error occurred: 'NSE_EQ|INE002A01018'

Please let me know why this is happening.

Thanks