Downloaded 13 times
Uploaded byBert Hajee
Easy oracle & weblogic provisioning and deployment
The document discusses the challenges of implementing DevOps practices in large organizations, particularly focusing on the integration of Oracle WebLogic and related technologies. It highlights the importance of automation and effective communication between development and operations teams to achieve efficient deployment processes. Using Puppet as a management tool, the narrative illustrates how to streamline configurations and foster collaboration, ultimately leading to quicker, more reliable application releases.
More Related Content
Similar to Easy oracle & weblogic provisioning and deployment
Easy oracle & weblogic provisioning and deployment
- 1. EASY ORACLE AND WEBLOGIC PROVISIONING &DEPLOYMENT
- 2. TODAY ENTERPRISE DEVOPS. CANIT BE DONE? AUTOMATION IS REQUIRED THE VALUE OF COMMUNICATION ORACLE AND WEBLOGIC CONFIGURATION HOW DOES IT WORK? WAT IS THE ISSUE? HOW TE GET STARTED?
- 3. INTRODUCTION • More then20 years experience in IT • As project manager & development manager at large IT companies and government • Founder of Enterprise Modules (twitter: @enterprisemodul) • (Co)developer of Puppet modules for Oracle RDBMS en WebLogic • bert.hajee@enterprisemodules.com • twitter: @bhajee
- 4. MISSION STATEMENT Provide ourcustomers with high quality Puppet modules for all the Enterprise products. Thereby allowing them to exploit all advantages of Puppet not only for their base OS and open source products, but for their entire Enterprise Infrastructure https://www.enterprisemodules.com twitter: @enterprisemodul
- 5. FINALLY... John looked atthe ceiling. Yesterday, after more than one and a half years of preparation, the latest version of the application went live. But it took a lot of effort and sparked a lot of negative energy. The application was a registration application that used Oracle WebLogic, and stored all data in an Oracle database. Before getting the application into production, the development team and the operations team where coastally at each other’s throat. Every time an issue was reported, during the tests, they were pointed fingers at each other and calling names. The operations guys blamed the developers. Saying the only delivered bad code. And that in the end the operations guys would be called in the middle of the night to clean up the mess. The developers, on the other hand, said that the operations team was just splitting hairs and fretting about non important issues. They were just stalling the project. If the operations team hadn’t been so fearfull of changes, the would already be running in production for a long time. When there was a performance problem, the DBA-er told the developer that he was not using the indexes correct. But the developers blamed the operations team for not providing a correctly configured operating system. Does anybody recognize these stories? These are stories we often hear from customers. Although the word DevOps has been buzzing in the world of startups, it has been tough to get it to work in larger companies.
- 6. ENTERPRISE DEVOPS?? Wouldn’t itbe great if we are able to do things in the devops-way? Wouldn’t it be great if John did not have to wait for one and a half years to get this new release into production? But actually do a new release every week. Or even do a release every day. Wouldn’t it be great if it didn’t matter if the change was in the application or in the underlying middleware database or operating system? The change would be done quick, consistent and painless. Wouldn’t it be great if the development team and the operations teams worked together in close cooperation towards attaining these goals.
- 7. Or is itjust impossible to get devops working in large companies…… Well…. I can tell you it is possible. In this presentation I will take you down the road that John and his team traveled. The steps they took to transform the current status que into a smooth operation where they were able to do multiple deployments per week and make controlled changes to the applications, middleware and operating systems. This story is based on actual experiences we had with multiple customers, both in the Netherlands and in other parts of Europe.
- 8. FAST RELIABLE & CONSISTENT AsI mentioned before, John and his team needed one and a half years to put the new version of the application into production. This time was divided over and his team had needed a half year to put the new version of the application into production. This time of course was divided on the various aspects and stages of such development process. What I did notice was that it proved difficult to put down a fast reliable and consistent environment. .... a lot of hand work ... Inconsistent … Sensitive to errors Small annoying errors labor-intensive Takes a long time
- 9. AUTOMATION REQUIRED For theteam things where clear. Automation of the work flow was required.
- 10. Application Problem?? WHAT IS THEPROBLEM??? But before we continue. Let’s see what the problem is. How can installing an application be a problem?
- 11. Application Middleware Problem? WHAT IS THEPROBLEM?? Well, one of the reasons is, you normally need a little bit more then just an application. Most enterprise applications need some sort of middleware.
- 12. Application Middleware Database Problem WHAT IS THEPROBLEM? And since data is important for large companies. Most of them need a database too.
- 13. Application Middleware Database OS Problem! WHAT IS THEPROBLEM And you would almost always need an operating system.
- 14. Problem!! Application Middleware Database OS Network WHAT IS THEPROBLEM! And stuff needs to be connected to a network. And as you can see slowly but surely the number of components that need to be installed and configured, increases.
- 15. Problem!!! Application Middleware Database OS Network Application Middleware Database OS Network WHAT IS THEPROBLEM!! Large companies most of the time, cannot afford downtime. So one of the risk mitigation counter measures used a lot it make the infrastructure high available by duplicating everything. And as you can see, the tasks of installing and configuring you IT infrastructure increasingly becomes a challenge.
- 16. Problem!!!! Application Middleware Database OS Network Application Middleware Database OS Network WHAT IS THEPROBLEM!!! But even this is manageable. But when we start to look at the the number of individual settings that need to be controlled, we start to see the real problem. The amount of stuff to manage is huge.
- 17. Problem!!!!! Application Middleware Database OS Network Application Middleware Database OS Network WHAT IS THEPROBLEM!!!! It is not only huge, it is also complicated. Some of the configuration points are interdepended. So when you change the size of the SGA, you also need to change the amount of memory reserved. When you change the port number of a weblogic server, you also need to change the firewall rules to allow communication to the specified IP port. And we start to see why it has been incredible difficult to get all of the systems in our chain of systems configured correctly.
- 18. A SOLUTION Like thefigured out before, they needed a tool. A tool that would help them manage these infrastructure layers and the huge amount of configuration points. Like the figured out before, they needed a tool. A tool that would help them manage these infrastructure layers and the huge amount of configuration points.
- 19. In the teamthe chose Puppet. Reasons for that where: • Communication • Large and stable player in the market • Big partners including Cisco, VMWare, Microsoft and EMC • Multi-platform including windows • Also for configuration of network and storage components • Support for configuration Oracle databases and WebLogic
- 20. Your infrastructure ona blueprint YOUR INFRASTRUCTUE IN A BLUEPRINT For the people who don’t know, let me explain how puppet works. When you are using using Puppet you describe your infrastructure in sort of a blue print. This blueprint contains a description of all the required components in your infrastructure, all the components on your servers. All the settings you need, are described in your blueprint. For example…
- 21. What’s reality? REALITY? After Puppethas read your blueprint, it goes to the system. It starts to make pictures of how the components are really configured and if they are installed at all. It makes an inventory of reality of the current system.
- 22. Spot the differences SPOTTHE DIFFERENCES By comparing the blueprint and the pictures, it can make a list of differences. And based on the list of differences, it can make a list of actions that need to be executed. For example: when your blueprint says it needs a datasource in domain ‘production’ in a WebLogic 12.2.1 environment, Puppet will look at its pictures and see no weblogic software is installed and obviously no datasource is configured. The puppet modules will have the knowledge of what the required actions are to install the software, create the domain and create the dataasource. If however an existing datasource is available, but just the url of the database has changed, puppet will see everything is ok and just change the url property of the datasource. What is really important, is tha puppet describes the end result and not the procedural steps needed to get there. Because depending on the begin situation, they might be different.
- 23. For new install’sand updates NEW INSTALLS & UPDATES This is a really handy mechanism. You can use this mechanism to provision a totally new system. Puppet will see a huge difference between the blue print and the pictures and will make a long list of actions to execute. But if there is just a small change, this mechanism also works. Just the list will be much shorter.
- 24. Blueprint for OracleTable space.. ooora_tablespace {'my_app_ts@sid': ensure => present, datafile => 'my_app_ts.dbf', size => 5G, logging => yes, autoextend => on, next => 100M, max_size => 20G, extent_management => local, segment_space_management => auto }, } NEW INSTALLS & UPDATES Here is an example of a part of a blueprint. This is a definition of an oracle tablespace. Key here is that you don’t have to be a database specialist to understand what is defined here. Later in the presentation I will come back to this.
- 25. ABSTRACTION But how doesthis solve the issue of the many many connected configuration points in our infrastructure? and the answer is: Abstraction
- 26. Abstraction Application Middleware Database OS Network Application Middleware Database OS Network In essencePuppet provides a consistent abstraction layer over the real infrastructure components. It allows you to make your infrastructure components into building blocks with a defined interface. Some of the building blocks are small and provide just a small configuration element. An example of this, is the ora_tablespace definition we saw before. But you can also put those small building blocks into a little bit bigger building blocks and define an interface for your standard configured weblogic domain, or your standard application setup on your database.
- 27. Separation of ConcernsApplication Middleware Database OS Network Application Middleware Database OS Network DEV MYAPP 10.10.10.100 Queue1 APP DEV ns1.dev.com username password mw_home db_home wls_server db_size em_size webserver db_server db_server Anotherpart of the way Puppet helps you is by separating concerns. Separating the what from the how. Most of the times we need multiple kind of setups. We need a setup for development, we need a setup of production and in between we need multiple setups for regular testing and acceptance testing. For the most parts, these setups are the same. But porbaby you need different IP setting in testing then in production. And maybe we can do with a little bit less space in the database. Puppet has a mechanism for allowing you to separate those settings from the rest. It is called hiera. Using hiera you can connect certain settings in your infrastructure to values in your hiera key-> value lookup. Here you see a setup where we have “wired" our setup for a development environment.
- 28. For different EnvironmentsApplication Middleware Database OS Network Application Middleware Database OS Network TEST MYAPP 10.10.10.100 Queue1 APP TEST ns1.dev.com username password mw_home db_home wls_server db_size em_size webserver db_server db_server Becausewe have extracted the information, it becomes really easy to transform this blueprint into a blueprint for a testing environment. We just have to adjust the specified settings and we can use the same blueprint with a different layover to manage our testing environment.
- 29.
- 30. Finally in ProductionApplication Middleware Database OS Network Application Middleware Database OS Network PROD MYAPP 10.10.10.100 Queue1 APP PROD ns1.dev.com username secret mw_home db_home wls_server db_size em_size webserver db_server db_server And of course also our production environment.
- 31. DIFFERENT KIND OF ENVIRONMENTS WhenJohn and his team started with puppet and the environment, things worked well. Finally they where able to consistently provision and deploy systems and applications. The started with a Puppet blueprint for production, then the extracted the properties that needed to be different for acceptance and put those into hiera. They gained a lot. But something was missing…. They noticed that a lot of issue still where only discovered when the application was deployed into the acceptance setup. There was still to big of a gap between what the developers used on there development workstations and what was running in production. What they wanted was to provide all developers with a minified version of the production environment on there own workstation. A setup that didn’t need to be high available and high performance, but when looked at from the application, provide the same interface as a production environment. The same queue’s, the same dataseource, the same tablespaces and the same tables. Just the clustering (for HA) and the sizing would be different.
- 32. PORTABLE PLATFORM An theymanaged to get that working. It is called the portable setup. Now they can use the same puppet blueprint to: - install a minified version of the setup on a PC. This contains both the database, the weblogic domain and loadbalancers. - Install a test setup where the database is on one machine and the weblogic domain is on an other machine - And a HA setup where they use a RAC oracle database and a 10 node WebLogic cluster to run the application on. John and his team noticed that a lot of the late issues disappeared. Because the developers had access to the same setup as used in production, a lot of setup issues like missing tablespaces, differently named queue’s and datasources, would be found much earlier in the development process.
- 33. PLATFORM CI Platform Code ApplicationCode Because of this easy portable setup, they were able to get two interconnected CI environments.
- 34. PROCES CHANGES Because ofthis interconnected CI environments, they had a consistent view of the quality of the platform and application. So they became pretty confident. At any moment they were sure they could provision and deploy the latest combination of working platform with a working application. Based on this confidence, they started to change their processes. Every week they would release a version of their platform puppet code and there application. Depending on the customer’s needs, they would just use this in their development chain, or deploy this to production. With this process they achieved agility while retaining stability. With this mechanism and process they were able to deploy new versions of the application, provision changes in the WebLogic and Oracle setup and provision patches, both on the OS as well as in the middleware layer.
- 35. COMMUNICATION AND COOPERATION Automating theirsetups sure helped. But John and his team where certain tooling was not enough to overcome the conflicts and heated discussions between the developers and the operations guys. Something had to be done to improve the collaboration. When you look at the literature, one of the key enablers of collaboration, is communication. Many development teams use a Kanban board to enable communication about the status of the project. The scrum methodology uses standups for communication and mandates that a team has all required skills and knowledge to do everything needed from idea to delivery.
- 36. A BIG TEAM Basedon the scrum methodology, they thought about having one big team. A team that would contain all required skills and be responsible and accountable for the whole process. But they noticed it didn’t work. The amount of different IT skills is large in large companies. If you would put all those skills in one team, the team would become huge. Also because of corporate policies it is hard to get all authorizations into that one team. Large companies often require separation of duties and authorities. So… Although in theory very nice. Not practical for large companies.
- 37. COMMON LANGUAGE But whileJohn and his team where looking at solutions, they noticed the amount of conflicts was decreasing. There were still heated discussion, but they seemed to be targeted at some specific setting. The use of Puppet seemed to have a huge positive impact on communication as well.
- 38. FACILITATES TARGETED DISCUSSIONS Thesimplicity of the Puppet language provided a common language between the development team and the operations team. No longer were they fighting over some unseen settngs somewhere, but they were discussing the pro’s and con’s of certain settings.
- 39.
- 40.
- 41.
- 42. ORA_CONFIG module • ora_asm_diskgroup •ora_asm_volume • ora_database • ora_exec • ora_init_param • ora_listener • ora_object_grant • ora_record • ora_role • ora_schema_definition • ora_service • ora_tablespace • ora_user • AND MORE ORA_INSTALL module • db_control • db_listener • db_rcu.rb • db_opatch • installdb • installem • install_emagent • installs • opatchupgrade • tnsnames • net • goldengate • client • autostartdatabase FOR ORACLE WE HAVE GOT
- 43. WLS_CONFIG module • wls_authentication_provider •wls_cluster • wls_datasource • wls_deployment • wls_domain • wls_jms_queue • wls_jms_topic • wls_messaging_bridge • wls_role • wls_saf_imported_destination_object • wls_saf_remote_context • wls_server • wls_workmanager_constraint • AND MUCH MUCH MORE WLS_INSTALL module • wls_install::bsu • wls_install::cluster_node • wls_install::domain • wls_install::fmw • wls_install::managed_server • wls_install::nodemanager • wls_install::opatch • wls_install::packdomain • wls_install::software • wls_install::storeuserconfig • wls_install::utils::fmwcluster • wls_install::utils::fmwclusterjrf • wls_install::utils::oimconfig • wls_install::utils::webtier • … AND MORE FOR WEBLOGIC/FUSION WE HAVE GOT
- 44. HOW TO GETSTARTED?
- 45.
- 46. BUSINESS AS USUAL Johnwas staring at the ceiling. He was just thinking back about a couple of months ago. Then they needed months, or even more than a year for a new version in production. And not only that, but these times would be filled with conflicts and over heated arguments, But not anymore. They were able to put a release into production any time. It doesn’t matter if it is a functional change to the application, of changes in middleware, database or OS. Although in case of an emergency, they were able to a release any time, the customers most of the time didn’t need this. So they adjusted their process to a release cycle of a small changes every week and once a month they did a larger release. For the larger release they had more personnel standby for problems, but until now the larger release had been just as stable as the smaller releases.