Separation of Service Interfaces from it's implementation

My career in IT started with Microsoft .NET technologies.

And over time, my observation was that technologies provided by Microsoft under the developer umbrella were stream lined, well integrated, tested and well-proven. Along with the build tools, IDE (editor), etc. it was very useful to have best practices available.

When my usage of .NET technologies reduced over time. (Have been a JavaScript fan since IE5 days, so NodeJS was inevitable) a big problem I realised was the non-availability of standard best practices in other developer technologies. There were many ways of doing stuff, each better or worse than the other. And the only way of determining which one works for your use case / team / product was by attempting a Proof Of Concept (PoC'ing) and <strikethrough>praying</srikethrough> assuming that all your requirements are documented as goals for the success of the Proof Of Concept

One Such Instance

While working on the architecture of our product platform my team at Litmus World exposed functionalities as services over HTTP(S). These were hosted on NodeJS using the MEAN Stack as they were primarily I/O calls. During the development phase, one struggle we faced was availability (or the non-availability) of best practices on exposing API / services over HTTP(s). (it sounds simple until it stops sounding simple.)

Over time we observed reports of breakages faced by consumers of these services. This, we realised was because the API parameters (request / response parameters) were being modified with no concern of backward compatibility. This started becoming a regular occurrence as our company started on-boarding new customers with requirements which resulted in modifications of these services. Problem we realized was that developers were not able to do impact analysis to ensure backward compatibility. This was because they did not have ability to document and validate request / response parameters. Further these parameters were defined as and when necessary in arbitrary locations in the code. The flexibility provided by NodeJS/ExpressJS was causing a maintainability problem. Especially with fresh developers who were still getting accustomed to the coding practices culture of our company.

We realised that we could fix this problem by having a Separation of Concerns (Segregation of Duties) pattern. The request / response parameter modifications should be controlled by a team constituting of senior members and members of the architecture team. Modifications should be reviewed and approved by this team, one checkpoint being the assurance of backward compatibility. The impact analysis should not be left to developers, whose primary goal is to get the service ready for our customers.

That is when we decided to bring in a practice which .NET framework had taught us. In .NET when exposing web services (such as ASMX services and WCF Services), we always separated the Service Interface and Service Implementation. This enabled us to version and modify the actual implementation and ensuring that the consumers faced fewer breakages. Interfaces had to be owned and controlled by Service Designers, and the implementations by the Engineering Team.

In NodeJS, ExpressJS one of the ways of implementing this solution was to utilize Swagger (https://swagger.io/). Swagger utilizes OpenAPI specifications (https://www.openapis.org/) to define the Service Interface. Further it provides a run-time which can be embedded into ExpressJS to ensure that the request and response adheres to the interface defined by the API Definition Team. The runtime also provides endpoints to expose documentation of the API services based on the interface defined and provided us ability to version our APIs.

Using Swagger solved one part of our problem. It helped us split our API Services into Interface and Implementation. We now had separate code files for each of these. We ensured a checkpoint in our code review process, to ensure that before merge of any feature the reviewer had to have a green tick from the API Defn team, if interface files were a part of the changeset. Though this was a solution to the problem, there still existed the problem of manual errors during code reviews, which may result in an approval of a changeset even though an interface was modified but the API Definition Team had not approved.

So to ensure complete enforcement, our team next separated out the source control repos for the Service Interfaces from the Implementations. Interfaces were put into a repository which allowed only Pull Requests from Developers. API Definition Team had merge responsibility after approval on the Interface Repository. After this a customized runtime was built. This customized runtime on build orchestrated the retrieval of the interfaces and implementation / controllers from the corresponding repositories and wired them.

This reduced the breakage of our exposed APIs by a good percentage.

Our next goal was to put metrics around usage of these APIs to ensure that dependency on consumers are captured. Will probably be another post. Interested? Subscribe to follow on.

This is just one hint of what my team, The Architecture, Platform and Infrastructure team does in Litmus World. Of course, we are hiring. You have the right attitude and skillset? And are interested in joining my team? Check out our careers page. And send out your resume to <view source>