-
Notifications
You must be signed in to change notification settings - Fork 101
Description
Java grpc does not have keepalive configuration by default. It will use the kernel tcp default configuration as the values
sysctl -a | grep tcp
net.netfilter.nf_conntrack_tcp_timeout_max_retrans = 300
net.netfilter.nf_conntrack_tcp_timeout_unacknowledged = 300
net.ipv4.tcp_keepalive_time = 7200
Vals in seconds
Why Keepalive in GRPC Client
Keepalive is recommended to use when your peer dies before it is able to notify you or does not notify you. The reasons can be kernel panic, termination of the process handling that pee
Hence, if a server does not send GOAWAY to the client, the grpc client will send packets to the TCP peer but the TCP perr will not send any packets. This can happen when a server nodes goes down due to machine failure, release/rollouts.
As you noticed the tcp (unacknowledged , keepalive) times are in 5 mins and 2 hours, so the reasonable solution will be to add keepalive logic in client library.
Steps to reproduce
To reproduce the problem, run the following command on a gRPC client host:
iptables -I INPUT -s <server-ip> -p tcp --sport <server-port> -j DROP