Hello, I tried fetching the live market data of 8k+ instruments in one go and got the bellow error:
websockets.exceptions.PayloadTooBig: Uncompressed payload length exceeds size limit (? > 1048576 bytes)
I tried to traceback the error and went through protocol.py, framing.py and permessage_deflate.py files but didn’t understand anything. Request you to help me in finding out the maximum permissible number of instrument keys for fetching live market data via websockets connection.
Please keep the max count at 100 instruments at a time. We are internally discussing increasing the max count with the concerned team. We will inform you once we make progress on it.
Noted. Thanks for support!
Hello Pradeep,
As you suggested to subscribe to 100 instruments at a go so I am planning to run a loop that subscribes to instruments in batches of 100. I want to know is the server providing the WebSocket service have its own limitations or restrictions. Do you recommend adding delay in between the batches?
Hello, Pradeep I am using below code for subscribing to multiple instruments at a time.
# Function to create batches of instrument keys
def chunks(lst, n):
for i in range(0, len(lst), n):
yield lst[i:i + n]
# Function to subscribe to instruments
async def subscribe_to_instruments(websocket, instruments):
data = {
"guid": "someguid",
"method": "sub",
"data": {
"mode": "option_chain",
"instrumentKeys": instruments
}
}
binary_data = json.dumps(data).encode('utf-8')
await websocket.send(binary_data)
# Function to subscribe to all instruments in parallel
async def subscribe_to_all_instruments(websocket, instruments):
tasks = [subscribe_to_instruments(websocket, batch) for batch in chunks(instruments, BATCH_SIZE)]
await asyncio.gather(*tasks)
async def fetch_market_data():
try:
ssl_context = ssl.create_default_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE
configuration = upstox_client.Configuration()
api_version = '2.0'
configuration.access_token = access_token
response = get_market_data_feed_authorize(api_version, configuration)
async with websockets.connect(response.data.authorized_redirect_uri, ssl=ssl_context) as websocket:
print('Connection established')
await asyncio.sleep(1)
# Subscribe to all instruments in batches
await subscribe_to_all_instruments(websocket, instrument_keys)
while True:
message = await websocket.recv()
decoded_data = decode_protobuf(message)
data_dict = MessageToDict(decoded_data)
print(json.dumps(data_dict))
# Call the provided callback function with the data
# await callback(data_dict)
except websockets.exceptions.ConnectionClosedError as e:
logging.error(f"Connection closed unexpectedly: {e}")
except websockets.WebSocketException as e:
logging.error(f"WebSocket exception: {e}")
except upstox_client.ApiException as e:
logging.error(f"Upstox API exception: {e}")
except Exception as e:
logging.error(f"An unexpected error occurred: {e}")
I am getting below error:
ERROR:root:WebSocket exception: code = 1000 (OK), no reason
Request you to help me with this. Thank you!
BATCH_SIZE=100.
Since over the websocket we have the capacity to subscribe up to 100 instrument_keys.
fetching option chain will be considered one request of based or it will be based total number of instruments?
Fetching the option chain is a single request and is not dependent on the total number of instruments.
However, you can retrieve the option chain for only one instrument key, with the expiry date being an optional parameter.
if possible can you help me with a code sample for option chain in websocket?
do u need complete option chain but using v3 it will be a different method but trust me v3 is super fast
@ulta_pulta sure man, thanks in advance
You can refer to different WebSocket implementations provided by Upstox: Upstox API Documentation
• GitHub Repositories for Various Languages:
• Java SDK
• PHP SDK
These repositories contain sample implementations and guidelines to integrate WebSocket streaming with Upstox APIs.