Document serializers

app/serializers/README.md

+# Serializers
+
+This is a documentation for classes located in `app/serializers` directory.
+
+In GitLab, we use [grape-entities][grape-entity-project], accompanied by a
+serializer, to convert a Ruby object to its JSON representation.
+
+Serializers are typically used in controllers to build a JSON response
+that is usually consumed by a frontend code.
+
+## Why using a serializer is important?
+
+Using serializers, instead of `to_json` method, has several benefits:
+
+* it helps to prevent exposure of a sensitive data stored in the database
+* it makes it easier to test what should and should not be exposed
+* it makes it easier to reuse serialization entities that are building blocks
+* it makes it easier to move complexity from controllers to easily testable
+  classes
+* it encourages hiding complexity behind intentions-revealing interfaces
+* it makes it easier to take care about serialization performance concerns
+* it makes it easier to reduce merge conflicts between CE -> EE
+* it makes it easier to benefit from domain driven development techniques
+
+## What is a serializer?
+
+A serializer is a class that encapsulates all business rules for building a
+JSON response using serialization entities.
+
+It is designed to be testable and to support passing additional context from
+the controller.
+
+## What is a serialization entity?
+
+Entities are lightweight structures that allow to represent domain models
+in a consistent and abstracted way, and reuse them as building blocks to
+create a payload.
+
+Entities located in `app/serializers` are usually derived from a
+[`Grape::Entity`][grape-entity-class] class.
+
+Serialization entities that do require to have a knowledge about specific
+elements of the request, need to mix `RequestAwareEntity` in.
+
+A serialization entity usually maps a domain model class into its JSON
+representation. It rarely happens that a serialization entity exists without
+a corresponding domain model class. As an example, we have an `Issue` class and
+a corresponding `IssueSerializer`.
+
+Serialization entites are designed to reuse other serialization entities, which
+is a convenient way to create a multi-level JSON representation of a piece of
+a domain model you want to serialize.
+
+See [documentation for Grape Entites][grape-entity-readme] for more details.
+
+## How to implement a serializer?
+
+### Base implementation
+
+In order to effectively implement a serializer it is necessary to create a new
+class in `app/serializers`. See existing serializers as an example.
+
+A new serializer should inherit from a `BaseSerializer` class. It is necessary
+to specify which serialization entity will be used to serialize a resource.
+
+```ruby
+class MyResourceSerializer < BaseSerialize
+  entity MyResourceEntity
+end
+```
+
+The example above shows how a most simple serializer can look like.
+
+Given that the entity `MyResourceEntity` exists, you can now use
+`MyResourceSerializer` in the controller by creating an instance of it, and
+calling `MyResourceSerializer#represent(resource)` method.
+
+Note that a `resource` can be either a single object, an array of objects or an
+`ActiveRecord::Relation` object. A serialization entity should be smart enough
+to accurately represent each of these.
+
+It should not be necessary to use `Enumerable#map`, and it should be avoided
+from the performance reasons.
+
+### Choosing what gets serialized
+
+It often happens that you might want to use the same serializer in many places,
+but sometimes the intention is to only expose a small subset of object's
+attributes in one place, and a different subset in another.
+
+`BaseSerializer#represent(resource, opts = {})` method can take an additional
+hash argument, `opts`, that defines what is going to be serialized.
+
+`BaseSerializer` will pass these options to a serialization entity. See
+how it is [documented in the upstream project][grape-entity-only].
+
+With this approach you can extend the serializer to respond to methods that will
+create a JSON response according to your needs.
+
+```ruby
+class PipelineSerializer < BaseSerializer
+  entity PipelineEntity
+
+  def represent_details(resource)
+    represent(resource, only: [:details])
+  end
+
+  def represent_status(resource)
+    represent(resource, only: [:status])
+  end
+end
+```
+
+It is possible to use `only` and `except` keywords. Both keywords do support
+nested attributes, like `except: [:id, { user: [:id] }]`.
+
+Passing `only` and `except` to the `represent` method from a controller is
+possible, but it defies principles of encapsulation and testability, and it is
+better to avoid it, and to add a specific method to the serializer instead.
+
+### Reusing serialization entities from the API
+
+Public API in GitLab is implemented using [Grape][grape-project].
+
+Under the hood it also uses [`Grape::Entity`][grape-entity-class] classes.
+This means that it is possible to reuse these classes to implement internal
+serializers.
+
+You can either use such entity directly:
+
+```ruby
+class MyResourceSerializer < BaseSerializer
+  entity API::Entities::SomeEntity
+end
+```
+
+Or derive a new serialization entity class from it:
+
+```ruby
+class MyEntity < API::Entities::SomeEntity
+  include RequestAwareEntity
+
+  unexpose :something
+end
+```
+
+It might be a good idea to write specs for entities that do inherit from
+the API, because when API payloads are changed / extended, it is easy to forget
+about the impact on the internal API through a serializer that reuses API
+entities.
+
+It is usually safe to do that, because API entities rarely break backward
+compatibility, but additional exposure may have a performance impact when API
+gets extended significantly. Write tests that check if only necessary data is
+exposed.
+
+## How to write tests for a serializer?
+
+Like every other class in the project, creating a serializer warrants writing
+tests for it.
+
+It is usually a good idea to test each public method in the serializer against
+a valid payload. `BaseSerializer#represent` returns a hash, so it is possible
+to use usual RSpec matchers like `include`.
+
+Sometimes, when the payload is large, it makes sense to validate it entirely
+using `match_response_schema` matcher along with a new fixture that can be
+stored in `spec/fixtures/api/schemas/`. This matcher is using a `json-schema`
+gem, which is quite flexible, see a [documentation][json-schema-gem] for it.
+
+## How to use a serializer in a controller?
+
+Once a new serializer is implemented, it is possible to use it in a controller.
+
+Create an instance of the serializer and render the response.
+
+```ruby
+def index
+  format.json do
+    render json: MyResourceSerializer
+      .new(current_user: @current_user)
+      .represent_details(@project.resources)
+  nd
+end
+```
+
+If it is necessary to include additional information in the payload, it is
+possible to extend what is going to be rendered, the usual way:
+
+```ruby
+def index
+  format.json do
+    render json: {
+      resources: MyResourceSerializer
+        .new(current_user: @current_user)
+        .represent_details(@project.resources),
+      count: @project.resources.count
+    }
+  nd
+end
+```
+
+Note that in these examples an additional context is being passed to the
+serializer (`current_user: @current_user`).
+
+## How to pass an additional context from the controller?
+
+It is possible to pass an additional context from a controller to a
+serializer and each serialization entity that is used in the process.
+
+Serialization entities that do require an additional context have
+`RequestAwareEntity` concern mixed in. This piece of the code exposes a method
+called `request` in every serialization entity that is instantiated during
+serialization.
+
+An object returned by this method is an instance of `EntityRequest`, which
+behaves like an `OpenStruct` object, with the difference that it will raise
+an error if an unknown method is called.
+
+In other words, in the previous example, `request` method will return an
+instance of `EntityRequest` that responds to `current_user` method. It will be
+available in every serialization entity instantiated by `MyResourceSerializer`.
+
+`EntityRequest` is a workaround for [#20045][issue-20045] and is meant to be
+refactored soon. Please avoid passing an additional context that is not
+required by a serialization entity.
+
+At the moment, the context that is passed to entities most often is
+`current_user` and `project`.
+
+## How is this related to using presenters?
+
+Payload created by a serializer is usually a representation of the backed code,
+combined with the current request data. Therefore, technically, serializers
+are presenters that create payload consumed by a frontend code, usually Vue
+components.
+
+In GitLab, it is possible to use [presenters][presenters-readme], but
+`BaseSerializer` still needs to learn how to use it, see [#30898][issue-30898].
+
+It is possible to use presenters when serializer is used to represent only
+a single object. It is not supported when  `ActiveRecord::Relation` is being
+serialized.
+
+```ruby
+MyObjectSerializer.new.represent(object.present)
+```
+
+## Best practices
+
+1. Do not invoke a serializer from within a serialization entity.
+
+    If you need to use a serializer from within a serialization entity, it is
+    possible that you are missing a class for an important domain concept.
+
+    Consider creating a new domain class and a corresponding serialization
+    entity for it.
+
+1. Use only one approach to switch behavior of the serializer.
+
+    It is possible to use a few approaches to switch a behavior of the
+    serializer. Most common are using a [Fluent Interface][fluent-interface]
+    and creating a separate `represent_something` methods.
+
+    Whatever you choose, it might be better to use only one approach at a time.
+
+1. Do not forget about creating specs for serialization entities.
+
+    Writing tests for the serializer indeed does cover testing a behavior of
+    serialization entities that the serializer instantiates. However it might
+    be a good idea to write separate tests for entities as well, because these
+    are meant to be reused in different serializers, and a serializer can
+    change a behavior of a serialization entity.
+
+1. Use `ActiveRecord::Relation` where possible
+
+    Using an `ActiveRecord::Relation` might help from the performance perspective.
+
+1. Be diligent about passing an additional context from the controller.
+
+    Using `EntityRequest` and `RequestAwareEntity` is a workaround for the lack
+    of high-level mechanism. It is meant to be refactored, and current
+    implementation is error prone. Imagine the situation that one serialization
+    entity requires `request.user` attribute, but the second one wants
+    `request.current_user`. When it happens that these two entities are used in
+    the same serialization request, you might need to pass both parameters to
+    the serializer, which is obviously not a perfect situation.
+
+    When in doubt, pass only `current_user` and `project` if these are required.
+
+1. Keep performance concerns in mind
+
+    Using a serializer incorrectly can have significant impact on the
+    performance.
+
+    Because serializers are technically presenters, it is often necessary
+    to calculate, for example, paths to various controller-actions.
+    Since using URL helpers usually involve passing `project` and `namespace`
+    adding `includes(project: :namespace)` in the serializer, can help to avoid
+    N+1 queries.
+
+    Also, try to avoid using `Enumerable#map` or other methods that will
+    execute a database query eagerly.
+
+1. Avoid passing `only` and `except` from the controller.
+1. Write tests checking for N+1 queries.
+1. Write controller tests for actions / formats using serializers.
+1. Write tests that check if only necessary data is exposed.
+1. Write tests that check if no sensitive data is exposed.
+
+## Future
+
+* [Next iteration of serializers][issue-27569]
+
+[grape-project]: http://www.ruby-grape.org
+[grape-entity-project]: https://github.com/ruby-grape/grape-entity
+[grape-entity-readme]: https://github.com/ruby-grape/grape-entity/blob/master/README.md
+[grape-entity-class]: https://github.com/ruby-grape/grape-entity/blob/master/lib/grape_entity/entity.rb
+[grape-entity-only]: https://github.com/ruby-grape/grape-entity/blob/master/README.md#returning-only-the-fields-you-want
+[presenters-readme]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/presenters/README.md
+[fluent-interface]: https://en.wikipedia.org/wiki/Fluent_interface
+[json-schema-gem]: https://github.com/ruby-json-schema/json-schema
+[issue-20045]: https://gitlab.com/gitlab-org/gitlab-ce/issues/20045
+[issue-30898]: https://gitlab.com/gitlab-org/gitlab-ce/issues/30898
+[issue-27569]: https://gitlab.com/gitlab-org/gitlab-ce/issues/27569