Rabbitmq

From Mosuma
Jump to: navigation, search

High latency connection to rabbitmq

For clients connecting to a far away rabbitmq host with ping of more than 1 s, you may get this error:

  File "/usr/local/lib/python2.7/dist-packages/pika/adapters/blocking_connection.py", line 275, in _adapter_connect
    raise exceptions.AMQPConnectionError(1)
pika.exceptions.AMQPConnectionError: 1

Trick is to set socket_timeout=RABBITMQ_SOCKET_TIMEOUT, here I set it to 1000ms:

RABBITMQ_HOST = "zhaoyun"
RABBITMQ_PORT = 5672
RABBITMQ_VIRTUALHOST = None
RABBITMQ_USERNAME = "XXXX"
RABBITMQ_PASSWORD = "XXXX"
RABBITMQ_CONNECTION_ATTEMPTS = None # 5 required for high-latency clients
RABBITMQ_SOCKET_TIMEOUT = 1000 # required for high-latency clients

credentials = pika.PlainCredentials(RABBITMQ_USERNAME, RABBITMQ_PASSWORD)

pika_parameters = pika.ConnectionParameters(
    host=RABBITMQ_HOST,
    port=RABBITMQ_PORT,
    virtual_host=RABBITMQ_VIRTUALHOST,
    credentials=credentials,
    channel_max=None,
    frame_max=None,
    heartbeat_interval=None,
    ssl=None,
    ssl_options=None,
    connection_attempts=RABBITMQ_CONNECTION_ATTEMPTS,
    retry_delay=None,
    socket_timeout=RABBITMQ_SOCKET_TIMEOUT,
    locale=None,
    backpressure_detection=None)

Time to rebuild queue index

  • approx 30 mins (quad core i7-3820 64G RAM) for rebuilding indices on 32 million messages in queue