API conventions are important because they promote consistency, predictability, and ease of use, allowing developers to quickly understand the structure and functionality of the API. Essentially, good conventions (also known as best practices) make APIs more intuitive and lead to smoother integration processes.
RESTful principles
At Gusto Embedded, all third-party integrations are RESTful. We strive to improve the developer experience and reduce the TTL (time-to-launch) for new partners. That means inter-operability, ease of use and consistency are top priorities.
We decided to implement REST APIs because they work over HTTP, are inherently suited for the web and can be accessed globally without special network requirements. Statelessness makes APIs more predictable, as each request contains all the information needed to process it, eliminating dependencies on server-side session state. Additionally, RESTful APIs use standard formats for data exchange (like JSON or XML) ensuring high compatibility with a wide range of clients and platforms. All these principles enable us to create highly compatible and consistent APIs for our partners.
Error Handling
We follow several guiding principles when it comes to API errors. One of the most important is that our error responses must follow a standard format. By standardizing the shape of our errors we ensure that our partners can confidently and consistently parse them.
Our standard format can be summarized as:
- Error responses always return an array of objects
- Error objects must include the base fields: error_key, category and message
- The category field (and the associated status code) must be one of our pre-defined error categories
- Error objects can provide additional information via the metadata field
An error response would look like this:
{
"errors": [
{
"error_key": "base",
"category": "payroll_blocker",
"message": "Company or employee address could not be verified. Please ensure all addresses are valid.",
"metadata": {
"key": "geocode_error"
}
}
]
}Another guiding principle is that error messages must be aggregated, actionable and user-friendly. Aggregated means that we gather as many errors as possible and return them in the resulting array. This enables our customers to resolve all of the issues at once, avoiding the frustrating pattern of solving one error just to discover another one in the next API call (“whac-a-mole” validations).
Actionable means that our error messages always contain relevant information about the action that must be taken in order to fix the error. But this information can’t be cryptic nor assume a specific UI design/pattern, and that’s where the principle user-friendly comes into play. We want our error messages to be clear, precise and with enough context for the user to quickly understand and debug the issue.
An aggregated, actionable and user-friendly error would look like this:
{
"errors": [
{
"error_key": "first_name",
"category": "invalid_attribute_value",
"message": "First name is required"
},
{
"error_key": "last_name",
"category": "invalid_attribute_value",
"message": "Last name is required"
},
{
"error_key": "date_of_birth",
"category": "invalid_attribute_value",
"message": "Date of birth is not a valid date"
}
]
}
Rate Limiting
Rate limiting is an important consideration when designing an API because it helps ensure the stability, security, and fair use of the system. APIs often share resources with other parts of the system (e.g., databases, processing power) and excessive requests from one user or application could degrade performance for others. By capping usage to acceptable levels we can ensure a consistent and reliable experience for all users.
APIs also incur operational costs (e.g., compute resources, third-party service fees), so rate limiting helps control these costs and make it easier to predict and manage resource usage, aiding in scaling the system effectively. And, on the security side, rate limits help mitigate brute force attacks, credential stuffing, or probing attempts by restricting how often users or systems can make API calls.
Our rate limits are scoped on an application-user pair at 200 requests per minute and are enforced in a 60-second rolling window. Throttled requests will return responses with the HTTP status 429: Too Many Requests until a new window opens.
Following our error handling principles, a rate limited request will look like this:
{
"category": “rate_limit_exceeded”,
“message”: “Rate limit exceeded. Please wait a bit before trying again.”
“status”: 429
}
Idempotency
Idempotency is the property of certain API operations that ensures they can be executed multiple times without changing the result beyond the initial execution, provided the calls have the same parameters.
In distributed systems, network issues or timeouts are not uncommon and they can lead to duplicate requests. If an API operation is idempotent, the repeated request won’t cause unintended side effects (such as financial transactions, database updates, or API resource creation), ensuring reliability, predictability, and safety; all very important non-functional requirements for our business.
To implement idempotent operations we use an optimistic concurrency control. In this approach, each object returned from the Gusto API includes a version field that denotes which version of the object you hold. Versions are essentially snapshots of a given resource. When a client wants to modify or delete this resource, it must provide the version value and the server will check if the provided value matches the current version of the resource. If they match, the operation is allowed and the version is incremented. Otherwise, the operation is rejected, indicating that the resource has been modified since the last time you made a request and we will return an HTTP status code 409 Conflict. We’d rather have an extra HTTP request or two than employees receiving incorrect pay.
Pagination
When exposing large datasets through your API you must provide a mechanism to break them into manageable chunks. Pagination helps reduce the amount of data sent in a single response, enhances scalability and facilitates data navigation, making it easier for customers to retrieve only the data they need. There are several pagination strategies (offset-based, cursor-based, page number-based, keyset) but at Gusto Embedded most of our endpoints offer an offset-based pagination.
The following API call will provide the second page of five employees of the company identified by the UUID abc123:
https://api.gusto.com/v1/companies/abc123/employees?page=2&per=5Since pagination parameters are provided in the request, some metadata about the paginated collection will be returned in the response’s headers to inform about the current page, total number of pages, total number of records in the collection and number of records being returned per page.
X-Page: 3
X-Total-Count: 542
X-Total-Pages: 22
X-Per-Page: 25
In order to ensure more consistent results for the API endpoints that work with real-time data models, we support cursor-based pagination. Rather than a page and per parameter, cursor-based pagination will use a start_after_uuid and limit parameter.
The following API will return the next five records after the given UUID:
https://api.gusto.com/v1/events?starting_after_uuid=10ac74e7-d6f0-46c0-9697-8ec77ab475ba&limit=5And the metadata returned for this endpoint will indicate whether there is another page of results or not:
X-Has-Next-Page: true | falseAPI Versioning
At Gusto we’re committed to enhancing the developer experience by continuously improving our services and introducing new features through the Embedded API. More often than not, these improvements and features can be introduced in a backward compatible fashion. However, sometimes we need to introduce breaking changes. Here’s where our API versioning system enters into play.
We use a date-based API versioning strategy that allows us to release backward incompatible changes in new versions while ensuring existing versions continue to receive support and benefit from new features. Once a new version is released, it is guaranteed to receive support for at least 12 months.
Partners can specify the version they want to use via the X-Gusto-API-Version header in their API calls. If no version is specified, we use either the default API version set for their application or the oldest stable version. Additionally, we pass back the X-Gusto-API-Version in the response header as a way for callers to confirm whether their requested version was respected or not.
Nonetheless, we can’t support countless versions of our API because of the burden it would place on our operations. So, as we release new versions, we also deprecate the oldest ones, making sure we have no more than four stable API versions at any given time. Once a version has been marked as deprecated, it enters a 12-month sunset period, 6 months of full support followed by 6 months of limited support. After the 12-month sunset period, the version is retired and all API calls will return a 406 Not Acceptable error.
API version deprecation is crucial for optimizing our efforts to maintain system quality, work on new features, and reduce technical debt but we also understand the cost that our partners must incur to migrate to newer versions. That is why we try to communicate these deprecation timelines as broad as possible, including not only emails and Slack but also HTTP headers.
Our API returns deprecation headers to notify when a resource has been scheduled for removal and where to find more information about how to upgrade to more recent versions. We do this via Deprecation, Link and Sunset headers. We encourage partners to monitor these headers to stay informed about new features and enhancements and plan ahead to avoid breaking integrations.
Anti-patterns
One of the most common anti-patterns we will encounter when working with monolithic applications is coupling. To reduce coupling, we enforce boundaries between modules using packwerk and apply separation of concerns. All the domain logic (payroll, taxes, benefits, etc) lives in their respective modules and the only point of contact between modules is via the API they explicitly expose. As an extra measure, we implement pre-commit hooks and CI checks that report any violation to these scopes.
Another common anti-pattern we try to avoid is swallowing errors in the lower levels of the implementation logic. This behavior prevents us from providing enough visibility to the layers closer to the partners and makes it more difficult to return actionable and user-friendly information. Bubbling errors up to the highest level allows us to handle the exceptional cases more effectively and establish a more centralized approach to error handling. This becomes especially important when we are dealing with complex code structures.
Along the same lines, we try to stay away from using generic exceptions. Instead, we define specific exceptions that provide all the context and information needed about each error, making debugging easier and more efficient. This also facilitates the mapping sometimes required to translate internal errors into external errors.
With regards to endpoint naming, we avoid using verbs as they lead to redundancy, inconsistent naming conventions and violate RESTful principles. Instead, we leverage the HTTP method to convey the action (GET for retrieving, POST for creating, PUT for updating, DELETE for deleting) and use resource-oriented naming (nouns) in URLs.
Whenever we need to identify resources, sequence IDs are strictly forbidden. Sequential numbers make it easy for attackers to guess valid resource IDs, reduce the flexibility for scaling and can expose details about the internal structure of our system, so we use UUIDs to identify all of our resources.
Public Documentation
To deliver an exceptional developer experience, a comprehensive and well-maintained documentation is as important as a good API design. Great public documentation not only empowers our partners to integrate seamlessly but also reflects the quality and thoughtfulness of our API design.
docs.gusto.com is our centralized hub for external documentation and it covers both, application integrations and embedded partners. The API reference is automatically generated from our OpenAPI spec while the guides, recipes and other sections are managed manually by our Technical Solutions team.
Maintaining accurate and up-to-date documentation is not without its challenges. Even though we have achieved a very good level of automation in our documentation generation pipeline, our API client is not being generated yet from the OpenAPI spec. This exposes us to inconsistencies and gaps that might lead to confusion among our partners and requires a constant effort to ensure documentation reflects the current state of our APIs.
That is the reason why we are currently investing in processes and tools not only to autogenerate our API clients but also to generate our OpenAPI spec based on test cases (using Rswag) to ensure our public documentation remains a reliable resource for our partners.
Closing Thoughts
The conventions and practices outlined in this document represent our commitment to delivering robust, intuitive, and secure APIs. While these guidelines aim to establish a strong foundation, they are not exhaustive. The landscape of API design is constantly evolving, and so is our approach.
At Gusto Embedded, we continuously evaluate and refine our practices to address emerging challenges and improve the developer experience. Our goal is to build APIs that are not only functional but also a pleasure to work with.
We encourage collaboration, feedback, and innovation from our teams and partners to ensure that our APIs remain aligned with real-world needs. By adhering to these principles and maintaining a mindset of continuous improvement, we strive to make our APIs better every day.
