Describe the feature: Indexing performance

Elasticsearch version (bin/elasticsearch --version): 6.0.0-beta1

Plugins installed: none

JVM version (java -version): openjdk version "1.8.0_131"

OS version (uname -a if on a Unix-like system): Linux 4.9.32-15.41.amzn1.x86_64 # 1 SMP Thu Jun 22 06:20:54 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Host: i3.16xl EC2 instance with 31GB heap configured, indexing buffer size of 50%, and data.path spread across 8 disks.

Description of the problem including expected versus actual behavior: Indexing throughput is degraded by up to 50% compared to 5.5.2. Seeing hotspots in LocalCheckpointTracker.margSeqNoAsCompleted(), generetaSeqNo() etc. Following shows the results for different tracks :

Steps to reproduce:

1. Restrict the rally challenge of each track to the following:
   {
         "name": "append-no-conflicts-just-index",
         "description": "benchmark",
         "index-settings": {
           "index.number_of_replicas": 0,
           "index.number_of_shards": 8,
           "index.refresh_interval": "100s",
           "index.translog.flush_threshold_size": "4gb",
           "index.translog.durability": "async",
           "index.translog.sync_interval": "240s"
           },
         "schedule": [
           {
             "operation": "index",
             "warmup-time-period": 240,
             "clients": 48
           }
         ]
       }

2. Run the challenge against 5.5.2 and 6.0.0-beta1 versions.
   esrally --pipeline benchmark-only --track=nyc_taxis --challenge append-no-conflicts-just-index --target-hosts=XX.XX.XX.XX:9200