More Related Content
Similar to Getting to Know the Cassandra Codebase
![Threads 2x[1]](https://cdn.slidesharecdn.com/ss_thumbnails/threads2x1-111010042244-phpapp01-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
Getting to Know the Cassandra Codebase
- 1. Cozy with CassandraGettingto know the Cassandra CodebaseGary Dusbabek • Rackspace@gdusbabekCassandra Summit • Mission Bay Conference Center • San Francisco • 10 August 2010
- 2. OutlineCode ThemesStartup SequenceKeyClassesRead PathWrite PathStages & ThreadingBootstrap & StreamingTests & IDE considerationsAdding API methodsQuestions
- 3.
- 4.
- 5. Themes and PatternsSingletonsand statics
- 6. Themes and PatternsStages&Thread pools
- 7.
- 8. CassandraDaemonLoads configurationTransport initializationStorage(Keyspace initialization)CommitLogrecoveryStorageService.initServer()Initializes CassandraServerPasses it off to transport
- 9. CassandraServerImplements IDL interfacemethods (cassandra.thrift, cassandra.genavro)Good place to start diving when adding or troubleshooting API methods
- 10. ConfigurationDatabaseDescriptorVia CassandraDaemon.setup()Looks forconfig path, loads yamlDoesn’t spin anything upDefines system tablesKS and CF described by CFMetaData and KSMetaData
- 11.
- 12. Main ControllersEnd with*Service or *ManagerStorageService, MessagingServiceCompactionManager, HintedHandoffManager, StageManager, StreamInManager
- 13. StorageProxyPut & GetmethodsCollection of static methodsMerges local and distributed operationsTracks latencyExposed via StorageProxyMBean
- 14. StorageServiceinitServer()—Starts servicesRegisters verbhandlers (in MessagingService)Main event respondersRepository of replication strategies and TokenMetadataRing topology & token information
- 15. MessagingServiceVerb handlers residehereSets up socket listenersGateway for outbound messagesMS.sendRR()MS.sendOneWay()Inbound tooMS.receive()
- 16. Table & ColumnFamilyStoreAlsoRowMutationLow-level storage operationso.a.c.db.*SSTableLocal operations
- 17.
- 18.
- 19.
- 20. Reading StorageProxy localTable, ColumnFamilyStoreCFSMakeQueryFilterQuery MemtablesQuery SSTablesCoalesce in iteratorso.a.c.db packageo.a.c.db.filter
- 21. Reading StorageProxy remoteread commandResponsehandlerSend to remote nodes
- 22. WritingSocket->CassandraServerValidationConvert to Mutation(IDL object)Penalties!
- 23. WritingStorageProxyblocking/non-blocking mutate local/remotebranchRowMutationone ColumnFamily perColumnFamilyCollection of column modifications
- 24. WritingRM.apply->Table.applyWrite to CLIterateover RM CFsCFS.apply()Overwrites results on pre-existing column families
- 25. WritingRM is serializedinto a Message and sent to other nodesWaits for ACKs depending on CL
- 26.
- 27. StagesSEDAhttp://www.eecs.harvard.edu/~mdw/papers/seda-sosp01.pdfo.a.c.concurrent.StageManagerRead, mutation, stream,gossip, response, anti entropy, load balance, migrationThread pools that consume tasks
- 28. ThreadsMessagingService.listen() spawns thread.Eachincoming connection spawns a new short-lived thread (IncomingTcpConnection)Non-stream ops go to MS.messageDeserializerExecutor_Stream ops handled there.Anti-entropy repair
- 29.
- 30. <=0.6 BootstrapingA wantsdata, B has data.StreamingRequestMessage A->BHandled on B by StreamRequestVerbHandlerFor each range StreamOut.transferRanges()Flush, anticompactionStreamInitiateMessage B->A for each range transferMeanwhile, back on A…StreamInitiateVerbHandler gets the SIM from B, does some nesting.StreamInitiateDone A->BBack to B…StreamInitiateDoneHandler gets the SID from ACalls StreamOutManager.startNext() which sends a single file to AMessagingService on A picks this up and the file is streamed.Sstable is createdSTREAM_FINISHED A->BB gets rid of the file, calls SOM.startNext()
- 31. 0.7 BootstrappingA wantsdata, B has dataStreamRequestMessage A->BOn B, StreamRequestVerbHandlerIf single file, sends it.If range, StreamOut.transferRangesForRequest()Send next file (first will contain meta data about all files)On A, IncomingStreamReader.read() Data is received, sstable createdAck, request next file
- 32. TestsTestable & UntestableUnittestsant clean build testSystem testsant gen-thrift-pynosetests test/system/test_thrift_server.py
- 33. IDEConfiguration file mustbe in the classpathTreat as sourcelib vs build/libLog at debug
- 34. IDE-ea -Xms128M –Xmx2G-Dcom.sun.management.jmxremote.port=8081 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false-Dcassandra-foreground=yes -Dlog4j.configuration=log4j-server.properties-Dmx4jport=9081
- 35. Adding API methodsSamegoes for modifyingDefine method and structures in IDLinterface/cassandra.thriftRegenerate filesant gen-thrift-java gen-thrift-pyImplement methods in o.a.c.thrift.CassandraServerCreate a system test (tests/system/test_thrift_server.py)
- 36.
Editor's Notes
- #3 Talk about Cassandra processes and the classes that relate to them.
- #4 Data operated on,then passed off to another layer.Evident on R/W pathOnion analogy
- #5 DisjointCoupled when neccessary
- #6 Model of class/object designSuck it upNot a class project
- #15 ServicesGossiperMessagingServiceMigrationManagerBootstrapPreloaded cacheHandle to server-related tasks.
- #28 layers