Config is a new SaaS offering for managing configuration files. Created by Bien David in 2017, the company looks to simplify how teams store and access configurations used by systems, apps, modules, environments, and server instances. InfoQ spoke to the team behind Config to learn more about these problems are solved.
Config is a new SaaS offering for managing configuration files. Created by Bien David in 2017, the company looks to simplify how teams store and access configurations used by systems, apps, modules, environments, and server instances. InfoQ spoke to the team behind Config to learn more about these problems are solved.
The Config platform runs in both public and private environments, and supports properties stored within INI, XML, JSON, YAML, and TOML configuration files. It includes capabilities such as version control, change review workflows, search, and support for continuous integration and deployment.
Config is currently in private beta, and includes multiple pricing plans. The price increases as the customer adds more users, or wants features like webhook notification, on-premises support, or more supported apps per system.
What’s the challenge with managing configurations today? How does this work with existing solutions that teams have in place already? InfoQ reached out to Bien David to answer these questions.
InfoQ: What’s the primary problem faced by developers and sys admins when it comes to config file management? Config: I would say the primary problem is how to easily manage config files across all the environments. I’m not talking about system config files that are the same across servers, as those can be part of the server image or rsync’d. I’m talking about config files that change per environment, like applications that go through a Dev to Prod migration.
My answer doesn’t show you the complete picture though, as the problem depends on your situation and job function. So let me talk a little about the problems faced by three groups: programmers, sysadmins and managers.
Programmers. When you have an application that needs to be deployed on multiple environments (local, Dev, QA, Staging, Prod) and multiple instances (cluster, region, etc.), the problem is how to make sure everything is in sync. How do you add a new configuration entry that is common on all servers? How about adding a new environment specific configuration? Related to this, is the question of where configuration files are stored.
When programmers start a new application, especially on a framework we are unfamiliar with, configuration goes through a certain life cycle, partly because configuration is not a high priority. Things start off with values being hard coded, then they turn into constants, then move on to a configuration file, then to an external configuration file or made into system environment variables. Configuration files eventually mature with environment overrides, or get transformed into centralized configuration. Even at this mature state, there is no easy or standard way of managing configuration that differs across environments. There’s still the manual effort of synching configuration. There’s still coding involved specific to your application framework or configuration library. And the problem only grows as the application grows, as more servers are used, and as the team becomes bigger and more distributed.
Next is the question of where to store these configuration files. You can store the configuration files on the servers themselves, or store them as part of the source code repository, or store them in a separate configuration repository, or store them on a centralized configuration server. Regardless of where the configuration is stored, you still have to making sure common and environment specific values are correctly applied in the master copies, and deployed to the right locations. And let’s not forget the fact that some companies lock down Staging and Prod environments, making the process of modifying configuration take longer.
Sysadmins. Sysadmins view the configuration file problem differently from programmers. First, they have to manage OS and system configuration. They need to easily provision new servers and desktops, with all the standard software that comes with them, and with the necessary environment or server specific configuration. Second, they need to worry about application configuration files in locked down environments. For custom applications, this usually means configuration files with less documentation and with no help available online. In the best case scenario, the application developer is still part of the company and is available to guide them.
Managers. The third group, the group seldom discussed, are technical managers who worry about making sure the Prod server is running smoothly. They need to fix Prod issues fast, or better yet, prevent them from happening. Since the application and servers are identical, ideally at least, configuration then is what varies across environments. So this is one of the things usually checked when something works in one environment but not the other.
Have you ever experienced something like this? The technical manager is scrambling because something is broken in Prod. They want to review the configuration and the logs but don’t have access. This is an old application and the original developers are gone, so they hunt down the sysadmin. The sysadmin is not familiar with the application configuration, so they work together and locate the config file directory, compare with other configuration files, and use available documentation to make sense of it all. There should be a better way.
InfoQ: Is there a problem that those users don’t even realize they have right now? Config: Many programmers, myself included, believe in YAGNI. Without a standardized configuration file management solution, we will do what is easy and straightforward for the application platform or framework we are using. Configuration files are not usually high priority, so we end up with various approaches. This works well during development, but becomes a problem when projects mature, and we have to manage many applications on multiple servers, all with their own little quirks.
For sysadmins, configuration management is one of their primary job functions, and the popularity of IaC is a testament to this. I think sysadmins know the problem, but some are hindered by company adoption.
The last thing I want to mention is the move to disposable applications and containers. If you can effortlessly bring up a new application running on a new container, then the only thing you need to worry about is configuration. This makes configuration file management much more important.
InfoQ: Where did you see a gap in the available solution space that inspired you to create Config? Config: Great question. I have a configuration file problem which I haven’t found a straightforward solution for. Old questions in Stack Overflow without an easy to implement answer. 2017 blog posts complaining about the same problem. We are in the age of AI and my relatively simple problem still exists.
So how do you manage all your application configuration files across all your environments? Here are the solutions right now, with Config filling the gap between storing configuration files in source control, and accessing them from a centralized configuration server. Think of Config as hosted source control with an easy-to-use web interface designed just for configuration files. A deployment time dependency so no runtime overhead, but with the benefits of centralized and runtime solutions, because of its on-demand configuration generation.
Manual Source Control Config Microservice Centralized Server IaC
I won’t go into the pros and cons of each. I will however compare existing solutions with Config.
Manual vs Config – If you have a small app that will remain small, running on one or two servers, doing things manually works. The only reason I can think of for using Config is if you have a lot of these small applications. You can use Config as an application inventory, so you have an overall view of all your applications, and their corresponding configuration locations, values, and documentation. If you outgrow your small setup, manually doing things won’t scale.
Source Control vs Config – For example, this will be storing your configuration files in Git. I recommend a separate repository for configuration files and their environment overrides. Other common approaches include: 1) a configuration template as part of the source code repository that you need to customize, 2) prod environment only configuration in source code repository, 3) all configuration files, including environment overrides, in the source code repository. Config is built with environment variables at the core. If you use Git, configuration files are just plain text files, and you still have to manually manage the environment differences, and implement code (or use a library) to handle environment overrides. Config lets you manage a configuration file, where you can add a new common key that applies to all environments, or tab through the environments and make environment specific changes. Like Git, Config has versioning and you do a pull to get the configuration you need. Config however, only pulls the configuration file for the environment you asked for, generating the correct configuration on-demand. When compared to hosted Git, like GitHub, Config is better since your configuration files are always private. You can’t accidentally make it a public repository, or easily make a public clone. Other Config features not found in Git (or GitHub) includes review or approval workflows, at rest encryption, push deployment, and type safe editing.
Microservice vs Config – An example of this is Spring Cloud Config. Works really well if you are using Spring Boot. Not really Spring specific but you need to write your own client. Config is better in the following situations. You prefer a deployment time solution, and not manage another server and introduce another runtime dependency. You don’t want to make code changes, or looking for a solution that will work with third party applications with configuration files.
