JaVers: Automating Data Auditing

When developing applications, we often encounter the need to store information about how data has changed over time. This information can be used both to debug the application more easily and to meet design requirements. In this article, we will discuss the JaVers tool, which allows you to automate this process by recording changes in the states of database entities.

How does JaVers work?

The library’s operation is based on storing information about business data entities, in dedicated tables, in JSON format. JaVers allows you to store this information in databases such as MongoDB, H2, PostgreSQL, MySQL, MariaDB, Oracle and Microsoft SQL Server.

When using MongoDB, JaVers creates two collections to store audit data:

• jv_head_id – Collection for storing the document containing the last value of commitId
  
• jv_snapshots – This collection contains detailed information about changes made to a business entity as a result of a creation, update or delete operation.

On the other hand, if you decide to use one of the relational databases, JaVers will create the following tables:

• jv_global_id – A table used to store a unique identifier for each change
  
• jv_commit – A table containing information about the time and author of the data modification
  
• jv_commit_property – An additional table, which allows you to store additional information to the data from the previous table – e.g. the identifier and username of the user. ID and the name of the user who is the author of the changes
  
• jv_snapshot – This table stores information about which attributes of an entity have changed as a result of a given operation, along with the values of each attribute.

To provide appropriate algorithms for comparing different versions of objects, JaVers operates on several data types: Entity, Value Object, Containers and Primitives.

Indicating the appropriate data type for each class, can be accomplished in 3 ways:

• Explicitly – using the register…() method of the JaversBuilder class or adding appropriate annotations over the selected classes
  
• Implicitly – by relying on JaVers to automatically detect the type for a given class, based on the class hierarchy
  
• Defaults – by treating all classes as ValueObjects

It is worth noting that JaVers maps annotations from the javax.persistence package by default, which makes it unnecessary to add additional annotations when implementing our database entities.

Implementation

To start using the JaVers tool, we need to add the appropriate dependency to our project. Depending on which database we will be storing the audit data in, we have two options to choose from:

• https://mvnrepository.com/artifact/org.javers/javers-spring-boot-starter-sql
  
• https://mvnrepository.com/artifact/org.javers/javers-spring-boot-starter-mongo

For this article, we will use a PostgreSQL database, so we include the following dependency in the pom file:

<dependency>
<groupId>org.javers</groupId>
<artifactId>javers-spring-boot-starter-sql</artifactId>
<version>6.6.3</version>
</dependency>

The way the library works will be presented on the example of an application, for storing information about ongoing projects in the company. The structure and implementation of the database entities of our application is as follows:

Project.java

@Entity
public class Project {

    @Id
    @GeneratedValue
    private UUID id;

    @NotNull
    private String name;

    @NotNull
    @Embedded
    private ProjectDetails details;

    @OneToMany(mappedBy = "project")
    private List<Member> members = new ArrayList<>();

    @Builder
    @AllArgsConstructor
    @NoArgsConstructor
    @Embeddable
    public static class ProjectDetails {

        @NotNull
        private LocalDateTime startTime;

        @NotNull
        private LocalDateTime endTime;
    }
}

Member.java

@Entity
public class Member {

    @Id
    @GeneratedValue
    private UUID id;

    @NotNull
    private String name;

    @NotNull
    private String surname;

    @NotNull
    private String role;

    @ManyToOne
    @JoinColumn(name = "project_id")
    private Project project;
}

In the database, we should then have the following list of tables available:

Data auditing

To indicate which business data should undergo the auditing mechanism, we can use three ways.

The first is to mark the repository of the selected entity with the @JaversSpringDataAuditable annotation. After adding this annotation, calling any method to modify data on this repository will cause information about this action to be placed in one of the JaVers tables.

In our application, we will only add the annotation to the Project class repository, as shown below:

	@Repository
	@JaversSpringDataAuditable
	public interface ProjectRepository extends JpaRepository<Project, UUID> {
	}

	@Repository
	public interface MemberRepository extends JpaRepository<Member, UUID> {
	}

The second way, which allows us to call the auditing mechanism for data, is to use the @JaversAuditable annotation, which we can add above the selected method to modify the data. Examples of such a solution, look as follows:

Service layer method

    @JaversAuditable
    public void save(Project project) {
        //save project
}

Repository layer method

@Override
@JaversAuditable
S extends Project> S save(S entity);

The last way by which we can save audit data is by calling the commit method on an object of the Javers class, which we can use using the dependency injection mechanism. A code snippet showing this solution in practice, looks as follows:

@RequiredArgsConstructor
public class ProjectService {

    private final Javers javers;

    public void commitProject() {
		Project project = new Project();//for simplicity ommited passing parameters
		javers.commit("author", project);
    }
}

AuthorProvider configuration

Before using these methods in practice, we still need to specify who is to be the author of the recorded audit data. An example of a basic configuration to determine this information, is shown below.

 @Configuration
public class JaversAuthorConfiguration {

    @Bean
    public AuthorProvider provideJaversAuthor() {
        return new SimpleAuthorProvider();
    }

    private static class SimpleAuthorProvider implements AuthorProvider {
        @Override
        public String provide() {
            return "Freddie Mercury";
        }
    }
}

Auditing data in practice

In order to verify the discussed mechanism in practice, the insert and modify data methods were called, resulting in the generation of INITIAL and UPDATE type audit data. The results of calling each method, look as follows:

Adding new data

A method that inserts new business data:

    public void save() {
        Project.ProjectDetails projectDetails = Project.ProjectDetails.builder()
                .startTime(LocalDateTime.now())
                .endTime(LocalDateTime.now().plusDays(14))
                .build();

        Project project = Project.builder()
                .name("Project 1")
                .details(projectDetails)
                .build();

        Member member = Member.builder()
                .name("Brian")
                .surname("May")
                .role("guitarist")
                .project(project)
                .build();

        project.getMembers().add(member);

        projectRepository.saveAndFlush(project);
}

As a result of calling the above method, the following entries were generated in the jv_snapshot table:

It’s worth noting that despite the addition of the @JaversSpringDataAuditable annotation only over the Project entity repository, the Member entity data was also written.

By default, JaVers includes all nested models belonging to the audited entity. We can change this behavior by adding the @DiffIgnore annotation over the field we want to omit from the auditing mechanism.

In our case, the use of this annotation would look as follows:

@DiffIgnore
    @Builder.Default
    @OneToMany(mappedBy = "project", cascade = CascadeType.ALL)
private List<Member> members = new ArrayList<>();

Modification of existing data

The method that modifies previously inserted data, in turn, is:

   public void update(UUID uuid) {
        Project project = projectRepository.getById(uuid);
        project.setName("Live Aid");
        projectRepository.saveAndFlush(project);
}

Calling the above method, however, results in the generation of an additional entry, as shown in the table below:

Downloading audit data

Stored audit data can be retrieved from the repository thanks to APIs provided by the library, referred to as JaVers Query Language (link: https://javers.org/documentation/jql-examples/). The data returned by the available methods, can be presented in one of three forms:

• Shadow – contains the historical data of the domain object, which are reconstructed recreated from snapshots (snapshots)
  
• Change – represents the difference between the properties of two objects
  
• Snapshot – contains historical data of a domain object, represented as a property map with values

The following are code snippets to return data in each of the forms discussed. In addition, under each snippet, examples of the data returned after calling the method are presented.

Shadows query

Query:

 public String getShadow() {
        JqlQuery query = QueryBuilder.byClass(Project.class).build();
        List<Shadow<Object>> shadows = javers.findShadows(query);

        return javers.getJsonConverter().toJson(shadows);
}

Response:

  [
  {
    "commitMetadata": {
      "author": "Freddie Mercury",
      "properties": [],
      "commitDate": "2022-05-03T20:02:27.554",
      "commitDateInstant": "2022-05-03T18:02:27.554689400Z",
      "id": 2.00
    },
    "it": {
      "id": "37bfc44c-ab29-44b5-869c-b39b705bdfdf",
      "name": "Live Aid",
      "details": {
        "startTime": "2022-05-03T20:02:12.274037",
        "endTime": "2022-05-17T20:02:12.274037"
      }
    }
  },
  {
    "commitMetadata": {
      "author": "Freddie Mercury",
      "properties": [],
      "commitDate": "2022-05-03T20:02:12.343",
      "commitDateInstant": "2022-05-03T18:02:12.343208800Z",
      "id": 1.00
    },
    "it": {
      "id": "37bfc44c-ab29-44b5-869c-b39b705bdfdf",
      "name": "Project 1",
      "details": {
        "startTime": "2022-05-03T20:02:12.2740372",
        "endTime": "2022-05-17T20:02:12.2740372"
      }
    }
  }
]

As we can see, the returned response contains information about what different versions of the same object looked like. The returned data does not have, for example, dedicated information about which value has changed in a given version.

Changes query

Query:

   public String getChanges() {
        JqlQuery query = QueryBuilder.byClass(Project.class).build();
        Changes changes = javers.findChanges(query);

        return javers.getJsonConverter().toJson(changes);
}

Response:

  [
  {
    "changeType": "ValueChange",
    "globalId": {
      "entity": "io.devapo.javerssample.entity.Project",
      "cdoId": "37bfc44c-ab29-44b5-869c-b39b705bdfdf"
    },
    "commitMetadata": {
      "author": "Freddie Mercury",
      "properties": [],
      "commitDate": "2022-05-03T20:02:27.554",
      "commitDateInstant": "2022-05-03T18:02:27.554689400Z",
      "id": 2.00
    },
    "property": "name",
    "propertyChangeType": "PROPERTY_VALUE_CHANGED",
    "left": "Project 1",
    "right": "Live Aid"
  },
  {
    "changeType": "NewObject",
    "globalId": {
      "entity": "io.devapo.javerssample.entity.Project",
      "cdoId": "37bfc44c-ab29-44b5-869c-b39b705bdfdf"
    },
    "commitMetadata": {
      "author": "Freddie Mercury",
      "properties": [],
      "commitDate": "2022-05-03T20:02:12.343",
      "commitDateInstant": "2022-05-03T18:02:12.343208800Z",
      "id": 1.00
    }
  },
  {
    "changeType": "InitialValueChange",
    "globalId": {
      "entity": "io.devapo.javerssample.entity.Project",
      "cdoId": "37bfc44c-ab29-44b5-869c-b39b705bdfdf"
    },
    "commitMetadata": {
      "author": "Freddie Mercury",
      "properties": [],
      "commitDate": "2022-05-03T20:02:12.343",
      "commitDateInstant": "2022-05-03T18:02:12.343208800Z",
      "id": 1.00
    },
    "property": "id",
    "propertyChangeType": "PROPERTY_VALUE_CHANGED",
    "left": null,
    "right": "37bfc44c-ab29-44b5-869c-b39b705bdfdf"
  },
  {
    "changeType": "InitialValueChange",
    "globalId": {
      "entity": "io.devapo.javerssample.entity.Project",
      "cdoId": "37bfc44c-ab29-44b5-869c-b39b705bdfdf"
    },
    "commitMetadata": {
      "author": "Freddie Mercury",
      "properties": [],
      "commitDate": "2022-05-03T20:02:12.343",
      "commitDateInstant": "2022-05-03T18:02:12.343208800Z",
      "id": 1.00
    },
    "property": "name",
    "propertyChangeType": "PROPERTY_VALUE_CHANGED",
    "left": null,
    "right": "Project 1"
  }
]

In the above example, we can see that the change information is returned only for the fields from the Project class, omitting the values of the embedded ProjectDetails type.

The resulting response has listed information about both the creation of the object and the assignment of values for a particular field. For this reason, when calling a data query in the form of Changes, it is worth narrowing the search area by setting appropriate filters.

Snapshots query

Query:

public String getSnapshot() {
        JqlQuery query = QueryBuilder.byClass(Project.class).build();
        List<CdoSnapshot> snapshots = javers.findSnapshots(query);

        return javers.getJsonConverter().toJson(snapshots);
}

Response:

 [
  {
    "commitMetadata": {
      "author": "Freddie Mercury",
      "properties": [],
      "commitDate": "2022-05-03T20:02:27.554",
      "commitDateInstant": "2022-05-03T18:02:27.554689400Z",
      "id": 2.00
    },
    "globalId": {
      "entity": "io.devapo.javerssample.entity.Project",
      "cdoId": "37bfc44c-ab29-44b5-869c-b39b705bdfdf"
    },
    "state": {
      "name": "Live Aid",
      "details": {
        "valueObject": "io.devapo.javerssample.entity.Project$ProjectDetails",
        "ownerId": {
          "entity": "io.devapo.javerssample.entity.Project",
          "cdoId": "37bfc44c-ab29-44b5-869c-b39b705bdfdf"
        },
        "fragment": "details"
      },
      "id": "37bfc44c-ab29-44b5-869c-b39b705bdfdf"
    },
    "changedProperties": [
      "name"
    ],
    "type": "UPDATE",
    "version": 2
  },
  {
    "commitMetadata": {
      "author": "Freddie Mercury",
      "properties": [],
      "commitDate": "2022-05-03T20:02:12.343",
      "commitDateInstant": "2022-05-03T18:02:12.343208800Z",
      "id": 1.00
    },
    "globalId": {
      "entity": "io.devapo.javerssample.entity.Project",
      "cdoId": "37bfc44c-ab29-44b5-869c-b39b705bdfdf"
    },
    "state": {
      "name": "Project 1",
      "details": {
        "valueObject": "io.devapo.javerssample.entity.Project$ProjectDetails",
        "ownerId": {
          "entity": "io.devapo.javerssample.entity.Project",
          "cdoId": "37bfc44c-ab29-44b5-869c-b39b705bdfdf"
        },
        "fragment": "details"
      },
      "id": "37bfc44c-ab29-44b5-869c-b39b705bdfdf"
    },
    "changedProperties": [
      "name",
      "details",
      "id"
    ],
    "type": "INITIAL",
    "version": 1
  }
]

In the example above, we can see that the Snapshot response contains a similar set of data as the Shodow response, but with the inclusion of a few additional elements, such as a list of changed values, or the type of audit data (INITIAL or UPDATE).