Have you ever wondered how software gets developed and eventually pushed out for the world to use? Ever wondered what #devops looks like from a developer’s perspective? There are many ways to develop software and avail the end product to users, and depending on the design and complexity of the software, engineers or software teams have developed workflows that help them produce better software.
Below is my workflow for developing Geldzin, my perpetually evolving web application. Apart from an IDE in which I do actual code development (Eclipse), the other major components in this #devops environment are Vagrant+Virtualbox, Bitbucket, Jenkins, Artifactory, and of course the pre-production and production servers.
(1) Most projects start with access to a source code repository. Always fork your own repository of the project in the remote host. Access assumes you have the necessary SSH keys registered in your profile to allow Git access via SSH.
(2) To have the source code locally available for development, initially clone the forked project repository off “master”. If there is already source code locally, setup the remotes needed and pull to synchronize local with remote.
(3) Whatever feature or issue you working on, checkout a local branch in which it will be developed. This allows you to work on a few different features or issues at a time, keeping the changes isolated until they are ready.
(4) I always have a virtual environment which I use to test or debug the application. I favor running applications within the IDE, but it is better to also test in a setup similar to production, with the expected dependencies in place (without security or load-balancing or caching).
(5) When you are done developing a feature or fixing an issue, merge the changes from the feature branch into the local “master” branch. Note that the local feature branches are not remotely linked.
(6) Resolve any merge conflicts and pull before you push the changes to your remote forked repository branch.
(7) Create a pull request, which triggers notification to other team members to review the code changes.
(8) The changes are voted on, or comments are added that may require additional code changes (putting you back at #3). Code should not be integrated until it has been approved by the team.
(9) I usually have a Jenkins job off my forked repository “master” that will build and run unit tests. Code should not be integrated until it has passed tests and coding standards. The test profile here should not require a VM; it should simply set off a sanity check.
(10) Integrating approved changes into the project’s “master” is usually a task for the team lead. At this point, I merge pull requests from other developers into the main “master”, and close the pull requests.
(11) Nightly, a Jenkins job will grab all the changes from the day, build the software, initialize a virtual environment, test the software, and publish reports or documentation. Any failures cause a jump back to #3 for the offending developer.
(12) If the nightly Jenkins job is success, code is automatically tagged with a version number and promoted to the “candidate” branch. Another round of testing can be done here if cherry-picking of unwanted features was done. Just remember to merge back to “master” (step #10).
(13) If all is well in #12, immediately another Jenkins job builds the software and posts the product in Artifactory with the appropriate version tag, and triggers #14 below.
(14) The Jenkins job in #13 above also deploys the software into a pre-production environment where quality analysis and stakeholder preview can happen. Any feedback from this point that needs changes causes a jump back to #3.
(15) At the appointed time for release, code is promoted to the “production” branch. This is our golden version that is released. I used to do build optimizations here, but I only generate production configurations/settings these days.
(16) The software is then deployed into production, and users have access to the latest and greatest. Notice that no further building of the software should happen here: the artifact from #13 should be imported instead.
(17) The process doesn’t end here for a developer: you usually need to update your development environment with all the changes from the latest release, so you fetch them from upstream.
(18) You finally merge the fetched changes into my local “master”, ready for the next feature or issue you will be working on. At this point, the cycle proceeds from #3 onwards.
As a developer, you always need to be aware of the distinct parts of the workflow: the #dev part is obviously what you do in development, and the #ops part is how your software ends up in operation. Knowing this forces you to design your projects for testability and externalize configuration settings, for the software to flow well in a #devops pipeline.