Learn about webhook delivery failures, message drops, and debugging processes to ensure the smooth operation of your webhook integration.
1. Webhook Handling
Delivery Failures
A webhook can fail to deliver messages if it receives a non-2xx status code for 10 consecutive delivery attempts. Each delivery attempt has an exponential backoff with a maximum period of one minute.
- Failure Handling: When delivery fails, the webhook's status is changed to "failed." No further messages are delivered until the verification process is completed. However, messages are still added to the target's topic to prevent data loss when reconnecting.
- Reconnection: Failed or active targets periodically receive a CRC request every 10 minutes. Upon successful verification, the target's status changes to "connected." A system message and backlogged messages will be sent to the target, followed by new messages as shown below.
{
"protocol": "system",
"event_type": "system.connection"
"data": {
"target_id": "ddfc2e56-f82c-4a24-975e-427230242476",
"disconnect_time": "2023-07-29T17:37:38.242292Z",
"connection_time": "2023-07-29T17:39:18.725464Z",
"previous_status": "failed",
"new_status": "connected"
}
}
Info:
The number of delivery attempts may be more than 10 in the event of a rolling upgrade of our services.
Message Drops
Each target’s topic has a maximum size that if exceeded because of slow processing or the target being unable to accept messages will result in older messages being dropped.
If messages are dropped, a notification is sent to the user as a "dropped message" event.
{
"protocol": "system",
"event_type": "system.dropped.messages",
"data": {
"dropped_msg_count": 1,
"pending_msg_count": 100,
"target_id": "85366adb-7550-400a-889f-3ae46f4b7668",
"time": "2023-07-29T16:09:31.045872283Z"
}
}
The current maximum number of messages is 10,000. The number of buffered messages per target can be controlled using the max_buffer_count parameter.
2. Debugging
Debugging Status Failed
If your target fails the verification check, its status will be "failed". You can debug the verification process using the /target/:id/validate-verification endpoint of the Config API.
Request Payload
Replace <TARGET_ID> with the webhook target ID you want to verify.
curl --location 'https://svc.blockdaemon.com/streaming/v2/targets/<TARGET_ID>/validate-verification' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--data ' '
Response Payload
{
"duration": "8.99986ms",
"error_response_message": "failed validation token is not as expected",
"error_status_code": 200,
"status": "failed"
}
The response includes the following information:
| Fields | Description |
|---|---|
status | The target's status based on the verification executed with this request. |
error_response_code | The response code returned by the webhook during the verification request. In this case, it returns a 200 - OK request. |
error_response_message | The details of what went wrong. In this example, the response token returned was invalid. This could be due to the secret used for the HMAC not matching the target's settings or the wrong payload used to create the HMAC. |
Debugging Processing Issues
To debug issues related to message processing, you can use the validate-message endpoint to send mock messages to your webhook. This allows you to simulate the messages your service will receive during operation.
Request Payload
Replace <TARGET_ID> with the webhook target ID you want to verify.
curl --location 'https://svc.blockdaemon.com/streaming/v2/targets/<TARGET_ID>/validate-message' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer token' \
--data '{
"protocol": "bitcoin",
"type": "confirmed_tx"
}'
Response Payload
{
"duration": "23.100988ms",
"status": "success"
}
Below is the example message that will be sent to the webhook target ID you specify in the request.
{
"protocol": "bitcoin",
"network": "mainnet",
"event_type": "unified_confirmed_tx",
"data": {
"block_number": 831386,
"block_hash": "0000000000000000000157bdde026859b310ba79f905f1514f022d1dd6129a00",
"tx_id": "77d80eb55bc1e99db7617c5b508452306e7049a54e7ec398a25a293339077763",
"tx_hash": "8dc10b4e32f1d0900e8786cd81875190533c2dc4310a360116409e26a700f19e",
"timestamp": 1233321795,
"status": "success",
"transfers": [
{
"asset": "native",
"from": "bc1qm34lsc65zpw79lxes69zkqmk6ee3ewf0j77s3h",
"value": 22349655,
"event_name": "vin"
},
{
"asset": "native",
"from": "bc1qu3aazsa92k2ws4mu5yzym0gwvdxvdqjmznmjgz",
"value": 582900,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qm34lsc65zpw79lxes69zkqmk6ee3ewf0j77s3h",
"value": 4215594,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qmwjjeestzkl260sf0zaxcylrjh3krjxmcatst7",
"value": 1376000,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qpm3ptrup3lme3dqj30zq3sy54mxuwgta8r5f99",
"value": 174060,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qagfwr2yl47ja4gmcu8yqqwx7cppmspdh3tjjaj",
"value": 589630,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qgzyta95e4tjkks900j3nzydp732ya6r8z4quv3",
"value": 7680000,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qzvkgdx0uma7dkcu4uull5sufm2lnmqf4dgyylf",
"value": 407180,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qp6qe3c7sgr9tzx6vjpjjln9hnanlsmxduazgwl",
"value": 374100,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1pf687nsnhxkynfzs2yt0ertedpjnj6enuqgufyc8qfymcgl8fzn5qdxvg0y",
"value": 980000,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qlv7kt6x44xp3xqsyx9896mr68sfs6zvq9ctq70",
"value": 1939139,
"event_name": "vout"
},
{
"asset": "native",
"from": "3GWcpDTJshR9jq9uBufz3ka42pZHgZKe6v",
"value": 1080000,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qxrzs7r49zhg6nhpxntzl0u4en2svksw4ynfp8h",
"value": 497411,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qzxcunle4hems53uehszlq84fn5cm6ejekd7f0e",
"value": 2288371,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qjc0cejltt57te78akwsc5ckuz250czp20r8y52",
"value": 77420,
"event_name": "vout"
},
{
"asset": "native",
"from": "bc1qe8l0s5ktdhpnys388xtt5vd0em8j38h7ktse4q",
"value": 60000,
"event_name": "vout"
}
],
"fee": {
"asset":"native",
"value": 27850
}
}
}
👋 Need Help?
Contact us through email or our support page for any issues, bugs, or assistance you may need.