Organizing SharePoint projects – Our DTAP street

With a number of SharePoint 2010 projects running in production and a few projects that are currently in their second stage, I thought it was about time to write something about how we organized these projects. In this post I will describe how we applied DTAP in our projects. It took us a year and half and a few revisions to get where we are now. In this post I will describe which environments we have and how we use them. And here is the disclaimer: it is a setup that works pretty good in our organization. It might work in your organization, but it also can be complete overkill. It depends. Most important is to think about this before you build your street. And in this post I hope to give you some insights in our situation that might help you thinking about yours.

Thinking about how many environments and how to use them is important, because it is good for everyone to have a clear understanding of the purpose of each environment. Also make sure everybody understands the processes around each environment. Who’s doing deployments, who to call when it is down, where to find the latest build, things like that.

We started working with SharePoint 2010, doing an internal proof of concept project around knowledge management. We had good discussions on how to organize things in Visual Studio, how to deploy our software and how to manage versions. A year and a half later things have grown a bit and we needed to re-think our strategies to make the growth possible.

Overview

I start off with giving you a schematic overview of our street. A few things to note:

• Besides intranet projects we also started to run internet facing sites. We have a separate SharePoint production farm for those websites. And this also led to 2 acceptance farms, 2 test farms etc
• Multi server farms are showed as 3 servers. This is not the real topology of these farms. The real topology is out of scope for the purpose of this post. Just the fact that a farm is built using multiple servers is important.
• Every server is virtualized.

In the following paragraphs I will talk about each environment, starting with the Development and moving forward to Production. For every environment I will discuss the following subjects:

• Usage – The purpose of the environment. Who is using it, and what for.
• Topology– Single server or multi server.
• Installation and management– Who is installing the servers and who is responsible for maintaining them. How do we install SharePoint and tooling.
• Deployment – How are projects deployed to the environment.

Development (D)

Usage – Used by developers and architects to build software and technical proof of concepts. These servers are also used by (functional) designers as demo environments to talk to users about SharePoint and build functional proof of concepts. Most developers have a dedicated server. For the other roles, we share servers. In SharePoint – BI projects multiple project members share a development server to build and test reports and scorecards.

• Topology– All development machines are standalone farms. Our development machines are not dedicated for one of the farms. To be flexible when people change teams, new projects start and other projects end, we want to be able to switch our development servers from intra- to internet and vice versa.
• Installation and management– Initial Installation of the OS and SharePoint is done by our IT-Pro team. We have a set of PowerShell scripts that install SharePoint and prerequisites and all tooling that we need. Day to day management is done by the developers themselves. They are responsible for installing service packs etc. They also have a script to change their dev server from an intranet to an internet development server. Main difference are the service applications that are used and the service accounts, that are dedicated for the type of farm. Initially developers get a fully functional machine, with the SharePoint farm up and running. Creating web applications, site collections, etc. is the first thing they do in their projects.
• Deployment – Software and SharePoint configuration we build in our projects are deployed to the local farm through Visual Studio deployment and through our Project Installer. This is a set of PowerShell scripts that installs WSP’s, configures SharePoint and adds content. It probably is the subject of a future post.
  For the developers there is 1 simple ps1 file to kick off. This cleans up the machine and re-installs everything that is created by the whole team. This way front-end developers who don’t know anything about SharePoint are also able to do and test their work in a SharePoint environment.
  In projects that are in their second stage, we still use this procedure to install the project to dev machines. Developers are also responsible to build the upgrade path to their latest version, but this is not tested in the development farm. In the setup we currently have it is not very easy to go back to a specific, to test the upgrade path over and over again.

Test – Development (T-D)

Usage – T-D environments are test farms dedicated to a project team. Used by testers in that team to test the software and SharePoint configuration. Main reason to have a dedicated farm is that testers are no longer dependent on other projects that run in the same farm. They control everything themselves and decide on their own deployment schedule to this internal test server. Larger project teams have a dedicated T-D environment. Small projects use the B-D farm for this purpose. If projects have testers that build automatic tests, this typically is the environment they use to run the tests.

• Topology– All T-D machines are standalone farms. They are the same machines as development machines. This way we are flexible when setting up new teams. T-D machines can become D machines and vice versa. After a project team stops, D and T-D machines are recycled for new projects, either in the intranet or the internet farm.
• Installation and management– Same story as for development machines.
• Deployment – Our Project Installer is used to install software to SharePoint and configure the environment for testers to be used. Generally one of the developers in the team has the task to install a new version to this environment. Most teams do this every day. The developer gets the latest code from Team Foundation Server, builds and packages everything and runs a re-install on his own machine. This is to validate everything that is checked in the previous day is installable. If this leads to a working site, he/she runs the same re-install (1 big PowerShell script) on the T-D server. If something is wrong, the team has time to fix it and testers still have yesterday’s build to continue testing. This way testers always have a working test environment, with just a short break when the installer runs. We started off by always installing the daily build, but this turned out not to be a good idea. Having 3 testers asking every 10 minutes when they can start testing is not very good for your blood pressure .

Build – Development (B-D)

Usage – B-D environments are the build servers dedicated to a farm. We have 2 build servers, one for intranet projects and one for internet projects. We use these environments for 5 purposes: 1) Daily validation if all code builds (using TFS TeamBuild) and can be deployed. 2) Testing if everything works in a multiserver environment, e.g. we catch a lot of “it works on my machine” in this environment. And because of the daily build, we find it the next day after it is checked in, instead of weeks later in the test environment. 3) Integration testing – this is the first environment where all projects get together. We use this to test if everything keeps running after installing the daily update of every project. This currently is a manual process. In the near future we hope to use the automated tests built by the projects teams for this. 4) Internal testing by testers in smaller projects that do not have a dedicated T-D server. We first started off by using this server as the test environment for all projects. This caused too many missed testing hours, because one of the projects was still fixing their daily build. Projects were simply too dependent on each other. We fixed it by introducing T-D servers. 5) Validation by IT-Pro’s if projects that are delivered to T are properly installed and tested by the development teams.

• Topology– Both B-D farms are multi-server, with a single web frontend.
• Installation and management– Same story as for development machines. Our development department is also responsible for managing this environment, just like the development servers. The challenge here is that multiple project teams are dependent on this farm. They need to work together to ensure test environments are available when needed and coordinate fixing of their daily builds.
• Deployment – Our Project Installer is used to install software to SharePoint and configure the environment. This is done by kicking off the PowerShell script for the project installer. This is done using the Windows Task scheduler on the application server, after the TFS teambuilds for all projects are finished. We are currently using TFS2010 with the MSBuild based teambuild. We haven’t yet upgraded the teambuild processes to the new workflow based teambuild. One day we will move to the approach as described by Mike Morton and Chris O’Brien.

Test (T)

Usage – For our IT-Pro’s everything we have talked about until now it called Development. For them, the test environment is where it all starts. The environments are used for: 1) Admins use this environment to learn project specific installations. 2) Project teams use it to fine tune the installations before they get to production (we all sometimes miss some config changes we made at the project start and forgot to document….). 3) Admins have a test form that they go through after every deployment. They do basic tests to ensure the environment stays up and running if this goes to production. They check event logs, health analyzer, etc. 4) Project teams use this environment for demos and for testing by business users. Generally all projects are installed a few times to T before they go to A. 5) Test connections to back-end systems. Not every installation to T needs to go through to production. We can install versions on T to validate connections to back office systems.

• Topology– Both T farms are multi-server.
• Installation and management– SharePoint and everything it needs is installed to the servers by our scripts. Of course without the development tooling. Developers do not have access to this server. They just have read permissions to the diagnostic logging folder. Everything else is done by admins.
• Deployment – Our Project Installer is used to install software to SharePoint and configure the environment. This is done by an IT Pro. Installation is done based on release notes. When projects deliver for deployment to T, they also deliver this document. It describes all required parameters, prerequisites, how to run the Project Installer and, if needed, manual configuration steps. Projects themselves decide their deployment schedule to the test environment. Some do it every (scrum) sprint, some do it weekly and some do it after a number of sprints. It is the scrum master’s task to ensure a time slot and admin are available to do the installation.
  We have a special mode for our Project Installer; it does not just do every action, but asks whether or not to run the action. For admins this makes it easy to see what happens and find errors easier. Installations in T are done with the help of one of the developers.

Acceptance (A)

Usage – The acceptance servers are used by our IT-Pro’s as final validation before we go to production. For the A farms, deployments need to be without problems. It is the final repetition before P-installation. We try not to do functional testing in this environment. After installing to the A environment we ask the product owner, to approve the application (or hotfix) for installation to the production environment. If something goes wrong during installation, or the product owner decides, some changes need to made, we need to go back to the project team and go from B-D to T, back to A. In the ideal world there’s 1 A installation for every P installation.

• Topology– Both A farms are multi-server.
• Installation and management– SharePoint is installed to the servers by the samen PowerShell scripts, but with different parameter sets.
• Deployment – A deployments are done by admins, based on the release notes.

Production (P)

Usage – Not a lot to tell here. It’s what it’s all about. We do all of the above to get everything we create to these farms.

• Topology– Both production farms are multi-server.
• Installation and management– SharePoint is installed to the servers by the same PowerShell scripts, but with different parameter sets. The big advantage of these PowerShell scripts, is that it is way easier to ensure all environments in our streets have the same setup. So if we have a multi tenant farm, all our development machines are also running the same multi tenant setup. And if we need Performance Point, the development machines also have the PPS service application running. From a quality perspective this is pretty important.
• Deployment – Production deployments are done by admins, based on the release notes. By the time we get here, the deployments have been tested a number of times, so they should run without problems. Our goal is to automate as much as possible by using and continuously improving our Project Installer. This way there is very little manual configuration left by the time we get to the P-installation. This greatly improves the quality of our installations.