How do I troubleshoot the error "READONLY You can't write against a read only replica" after failover of my Redis (cluster mode disabled) cluster?

2 minute read
0

Why am I receiving the "READONLY You can't write against a read only replica" error in my Amazon ElastiCache for Redis (cluster mode disabled) cluster after failover?

Short description

If the primary node failed over to the replica nodes in your Amazon ElastiCache cluster, then the replica takes the role of primary node to serve incoming requests. However, in the following scenarios you receive the READONLY error:

  • You're using a node endpoint instead of primary endpoint of the cluster in your application.

-or-

  • DNS caching in the application node routes traffic to the old primary node.

Resolution

1.    Verify that the cluster is cluster mode disabled. To do this:
Open the ElastiCache console, and then select Redis clusters. Verify that the Cluster Mode for the cluster is off.
Note: If the Cluster Mode is on, see I'm using ElastiCache or Redis. Why are my Redis client read requests always read from or redirected to the primary node of a shard?

2.    Verify that you're sending the write command to the primary endpoint instead of the node endpoint. To validate that the write command is going to the primary node, do one of the following:
Option 1
Connect to the Redis cluster using the redis-cli and then run the get key command for the updated key. Then, verify the command output to verify that the key value updated after the last command.
For example, the following command sets the key1 value to hello:

set key1 "hello" 
OK

To verify that the key set to correctly, run the get command:

get key1
"hello"

Option 2

Connect to the Redis cluster using the redis-cli and then run command the MONITOR command. This lists all commands coming to the cluster. Keep in mind that running a single MONITOR client might reduce throughput by more than 50%.

3.    To avoid DNS caching issues, turn on retry logic in your application following the guidelines for the Redis client library that your application uses.


AWS OFFICIAL
AWS OFFICIALUpdated 2 years ago