python requests retry on connection error

Python Requests Retry on Connection Error

If you are working with Python requests library and face connection errors, then retrying the request can be a good solution instead of giving up. There are various ways to implement retry logic in Python requests. Here are a few:

Using Retry Library

You can use the Retry module from urllib3. It provides a lot of options to configure retry logic.

import requests
from urllib3.util.retry import Retry
from requests.adapters import HTTPAdapter

retry_strategy = Retry(
    status_forcelist=[500, 502, 503, 504],
    method_whitelist=["HEAD", "GET", "OPTIONS"]
adapter = HTTPAdapter(max_retries=retry_strategy)
http = requests.Session()
http.mount("https://", adapter)
http.mount("http://", adapter)

response = http.get("")

Using Backoff Library

The Backoff library provides a simple way to add exponential backoff retries to your code.

import requests
from requests.adapters import HTTPAdapter
import backoff

@backoff.on_exception(backoff.expo, requests.exceptions.RequestException, max_tries=5)
def get_data():
    session = requests.Session()
    session.mount('http://', HTTPAdapter(max_retries=3))
    session.mount('https://', HTTPAdapter(max_retries=3))

    response = session.get('')
    return response.json()

data = get_data()

Using Retry-Session Library

The Retry-Session library provides a simple way to add retry logic to your requests by just creating a session object.

from retry_session import RetrySession
session = RetrySession()
response = session.get("")

These are some of the ways you can implement retry logic in Python requests. Choose the one that suits your needs the best.