Nginx Reverse Proxy with Kafka.pptx

Nginx Reverse Proxy with Kafka.pptx

• 2.
  Intro Kafka Cluster incoming message -without kafka client library support (e.g. Android or IOS app, IOT devices, …) - send message using Rest APIs
• 3.
  why we usenginx? source : https://www.nginx.com/resources/wiki/community/why_use_it/
• 4.
  Sample architecture Kafka Cluster upstreamtestme{ server backendAddr:8090 server backendAddr:8091 server backendAddr:8092 } incoming message http://[nignx server address]:8070/api backendAddr:8090 backendAddr:8091 backendAddr:8092 Load Balancing … (can be added)
• 5.
  Install Nginx • vi/etc/yum.repos.d/nginx.repo • yum install nginx • firewall-cmd --permanent --zone=public --add-port=80/tcp • firewall-cmd --reload [nginx] name=nginx repo baseurl=https://nginx.org/packages/centos/$releasever/$basearch/ gpgcheck=0 enabled=1
• 6.
  Install Nginx • ConfigurationDirectory : /etc/nginx • Configuration Reload : nginx -s reload • Service Stop : systemctl stop nginx • Service Start : systemctl start nginx http://your server address
• 7.
  [Port] Setting Permissionand Open • semanage port -l | grep http_port_t • semanage port -a -t http_port_t -p tcp 8090 • semanage port -a -t http_port_t -p tcp 8091 • semanage port -a -t http_port_t -p tcp 8092 • semanage port -a -t http_port_t -p tcp 8070
• 8.
  Log • access log tail-f /var/log/nginx/access.log • error log tail -f /var/log/nginx/error.log
• 9.
  Configuration reference : https://www.sobyte.net/post/2021-09/nginx-proxy-remove-prefixes/ upstreamtestme { server 127.0.0.1:8090; server 127.0.0.1:8091; server 127.0.0.1:8092; } server { listen 8070; listen [::]:8070; location /api { proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_connect_timeout 300; proxy_send_timeout 300; proxy_read_timeout 300; send_timeout 3000; proxy_pass http://testme/api; } } # pwd /etc/nginx/conf.d # vi test.conf firewall-cmd --permanent --zone=public --add-port=8070/tcp firewall-cmd --reload nginx -s reload systemctl stop nginx systemctl start nginx
• 10.
  Backend Application • python3testBackend.py 8090 & • python3 testBackend.py 8091 & • python3 testBackend.py 8092 & testBackend.py source : https://github.com/wonyongHwang/KopoBlockchain/tree/master/ref
• 11.
  Test • curl --location--request POST 'http://192.168.56.30:8070/api' --header 'Content- Type: application/json;encode=utf-8' --data-raw '{"key":"value"}' request response application log nginx access log
• 12.
  Backend Application asa Producer (1/2) prerequisite : kafka-python (https://kafka-python.readthedocs.io/en/master/install.html) testKafka.py source : https://github.com/wonyongHwang/KopoBlockchain/tree/master/ref
• 13.
  Backend Application asa Producer (2/2) request response application log kafka-console-consumer.sh --bootstrap-server master:9092,slave1:9092,slave2:9092 --topic kopo- topic --from-beginning kafka consumer
• 14.
  Reference • https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/