Rego & Go

Rego is a declarative policy language (also a logic language) that is used in OPA. It’s a general purpose language that works in many scenarios that aren’t necessarily just for policies.

One pattern I’ve recently come to like is using Rego inside of my Go project if the business logic becomes more complicated while leveraging the power of Go’s HTTP libraries. We can either embed our Rego or dynamically load it in if we decide we need to easily make changes.

I’m going to take a simple example, but hopefully you can see how using this inside of Go can be a nice experience. Let’s build a system that takes in job applicants and verifies that the applicant meets the criteria for the position.

Validating In Go

Let’s write some Go to do our validation

var validCities = []string{"Pittsburgh", "Cranberry", "South Hills", "Allison Park"}

type Applicant struct {
	Name           string `json:"name"`
	Experience     int    `json:"experience"`
	City           string `json:"city"`
	Salary         int    `json:"salary"`
	EmploymentType string `json:"employment_type"`
	CurrentRole    string `json:"current_role"`
}

type Job struct {
	EmploymentType string   `json:"employment_type"`
	MinExperience  int      `json:"min_experience"`
	MaxExperience  int      `json:"max_experience"`
	BasePay        int      `json:"salary"`
	Roles          []string `json:"roles"`
}

func validateCity(city string) bool {
	for _, v := range validCities {
		if v == city {
			return true
		}
	}

	return false
}

func validRole(currentRole string, roles []string) bool {
	for _, v := range roles {
		if v == currentRole {
			return true
		}
	}

	return false
}

func (a Applicant) validateFit(j Job) []error {
	var errs []error

	if a.Salary > j.BasePay {
		errs = append(errs, fmt.Errorf("%s is too greedy", a.Name))
	}

	if float32(a.Salary) < (float32(j.BasePay) * 0.8) {
		errs = append(errs, fmt.Errorf("%s doesn't want enough, they're too desperate", a.Name))
	}

	if a.Experience < j.MinExperience {
		errs = append(errs, fmt.Errorf("%s does not have enough experience", a.Name))
	}

	if a.Experience > j.MaxExperience {
		errs = append(errs, fmt.Errorf("%s has too much experience, they'll leave right away", a.Name))
	}

	if a.EmploymentType != j.EmploymentType {
		errs = append(errs, fmt.Errorf("%s wants %s time instead of %s time", a.Name, a.EmploymentType, j.EmploymentType))
	}

	if !validateCity(a.City) {
		errs = append(errs, fmt.Errorf("%s does not live in the right area", a.Name))
	}

	if !validRole(a.CurrentRole, j.Roles) {
		errs = append(errs, fmt.Errorf("%s will have too hard of a time in this role", a.Name))
	}

	return errs
}

We are being nice here and returning reasons for the denials. In the real world this could be trimmed down because we all know the applicant would be denied and never find out why.

Now this is fairly simple. We just run these validations against the applicants. However if business decides that we need another rule, we now have to add another rule here and recompile our application.

Validating In Rego

Let’s look at what this would look like implemented in Rego.

package applications

import future.keywords.if
import future.keywords.in

valid_cities := ["Pittsburgh", "South Hills", "Cranberry"]
employment_types := ["full", "part"]

default approved := false

approved {
  count(denied) == 0
}

denied[msg] {
  input.applicant.experience < data.job.min_experience
  msg := sprintf("%s does not have enough experience", [input.applicant.name])
}

denied[msg] {
  input.applicant.experience > data.job.max_experience
  msg := sprintf("%s has too much experience", [input.applicant.name])  
}

denied[msg] {
	not input.applicant.city in valid_cities
  msg := sprintf("%s does not live in the right area", [input.applicant.name])
}

denied[msg] {
  input.applicant.salary < (data.job.salary * 0.8)
  msg := sprintf("%s doesn't want enough, they're too desperate", [input.applicant.name])
}

denied[msg] {
  input.applicant.salary > data.job.salary
  msg := sprintf("%s is too greedy", [input.applicant.name])
}

denied[msg] {
  not input.applicant.current_role in data.job.roles
  msg := sprintf("%s will have too hard of a time in this role", [input.applicant.name])
}

denied[msg] {
  not input.applicant.employment_type in employment_types
  not input.applicant.employment_type == data.job.employment_type
  msg := sprintf("%s wants %s time instead of %s time", [input.applicant.name, input.applicant.employment_type, data.job.employment_type])
}

response["approved"] := approved if  {
  approved
}

response["denials"] := denials if {
  count(denied) > 0
  denials := denied
}

Not much shorter, however it’s declarative and when the logic becomes more complex, we have more options to declare our ruleset. In the real world where we decide to not care about our applicants, it could be this simple without denial reasons:

package applications

import future.keywords.if
import future.keywords.in

valid_cities := ["Pittsburgh", "South Hills", "Cranberry"]
employment_types := ["full", "part"]

default approved := false

applicant := input.applicant

approved {
  applicant.experience > data.job.min_experience
    applicant.experience < data.job.max_experience
    applicant.city in valid_cities
    not applicant.salary < (data.job.salary * 0.8)
    not applicant.salary > data.job.salary
    applicant.current_role in data.job.roles
    applicant.employment_type in employment_types
  applicant.employment_type == data.job.employment_type
}

response["approved"] := approved if  {
  approved
}

To be fair, this would also be fairly simple in Go without the error messages.

The response object here is a convenience so we can return a response with the errors and the approval. Rego by default will return all decisions made, so this lets us format an object we can return that contains the data in the format we want.

I mentioned before that we can either embed this or live load it. Let’s take a look at how this is done.

Calling Rego From Go

Let’s assume our Rego is stored in a file called applicants.rego. Let’s assume we want to be able to edit the file while the service is running. To load it in, we can build the rego object like this when the app starts. We can also use embed to embed the rego policy from a file.

type Server struct {
	Rego *rego.Rego
}

func main() {
  s := Server{
    Rego: rego.New(
      rego.Load([]string{"applicants.rego"}, nil),
      rego.Query("x = data.regoinject"),
    ),
  }
  r := http.NewServeMux()
  r.Handle("/testing", handleErrors(s.handleRego))

  log.Fatal(http.ListenAndServe(":9090", r))
}

For this simple example, we are going to attach the Rego object to a server struct. Then we can implement a method on our struct for our handler. I’m just adding a simple wrapper to handle errors and I only set them to 500 because I’m lazy. This also returns a JSON response. Admittedly doesn’t make a ton of sense in this example, but it makes for a working example when everything is done.

type errorHandler func(http.ResponseWriter, *http.Request) error

func handleErrors(h errorHandler) http.HandlerFunc {
  return func(w http.ResponseWriter, r *http.Request) {
    err := h(w, r)
    if err == nil {
      return
    }

    w.WriteHeader(500)
    log.Println(err)
    w.Write([]byte(`{"error": "uh oh spaghettios"}`))
    return
  }
}

func (s *Server) handleRego(w http.ResponseWriter, r *http.Request) error {
  var input Request
  if err := json.NewDecoder(r.Body).Decode(&input); err != nil {
    return err
  }

  resp, err := s.doTheRego(r.Context(), input)
  if err != nil {
    return err
  }

  return json.NewEncoder(w).Encode(resp.Response)
}

Now lets get to the meat of this. Since we have our Rego object, we can now build our evaluator. A couple notes here.

I’m using a function with the super original name getData to simulate getting data from somewhere else. In this specific instance the data could technically be passed in through the input object, but I’m demonstrating how we dynamically load data with Rego.
Since creating the store is a functional option, we can create the function from rego.Store and then pass our Rego object into it to set the store.

func getData() string {
  return `{
    "job": {
      "max_experience": 15,
      "min_experience": 5,
      "salary": 100000,
      "employment_type": "full",
      "roles": ["engineer", "data_scientist"]
    }
  }`
}

func (s *Server) doTheRego(ctx context.Context, input Request) (Response, error) {
  stringData := getData()
  d := json.NewDecoder(bytes.NewBufferString(stringData))
  d.UseNumber()
  var data map[string]interface{}
  if err := d.Decode(&data); err != nil {
    return Response{}, err
  }
  
  f := rego.Store(inmem.NewFromObject(data))
  f(s.Rego)
  
  eval, err := s.Rego.PrepareForEval(ctx)
  if err != nil {
    return Response{}, err
  }
  
  results, err := eval.Eval(ctx, rego.EvalInput(input))
  if len(results) == 0 {
    return Response{}, err
  }
  if err != nil {
    return Response{}, err
  }
  
  return handleResponse(results)
}

func handleResponse(results rego.ResultSet) (Response, error) {
  var response Response
  resp, err := json.Marshal(results[0].Bindings["x"])
  if err != nil {
    return response, err
  }
  
  if err := json.Unmarshal(resp, &response); err != nil {
    return response, err
  }
  
  return response, nil
}

Putting It All Together

This is a runnable example. You can copy it and run it on your machine. Don’t forget to copy the Rego file above that includes the denials. While the service is running, you can modify the Rego file and they will immediately take effect.

package main

import (
  "bytes"
  "context"
  "encoding/json"
  "log"
  "net/http"
  
  "github.com/open-policy-agent/opa/rego"
  "github.com/open-policy-agent/opa/storage/inmem"
)

var validCities = []string{"Pittsburgh", "Cranberry", "South Hills", "Allison Park"}

type Server struct {
  Rego *rego.Rego
}

type Request struct {
  Applicant Applicant `json:"applicant"`
}

type Applicant struct {
  Name           string `json:"name"`
  Experience     int    `json:"experience"`
  City           string `json:"city"`
  Salary         int    `json:"salary"`
  EmploymentType string `json:"employment_type"`
  CurrentRole    string `json:"current_role"`
}

type Job struct {
  EmploymentType string   `json:"employment_type"`
  MinExperience  int      `json:"min_experience"`
  MaxExperience  int      `json:"max_experience"`
  BasePay        int      `json:"salary"`
  Roles          []string `json:"roles"`
}

type Response struct {
  Response struct {
    Approved bool     `json:"approved"`
    Denials  []string `json:"denials,omitempty"`
  } `json:"response"`
}

func getData() string {
  return `{
    "job": {
      "max_experience": 15,
      "min_experience": 5,
      "salary": 850000,
      "employment_type": "full",
      "roles": ["engineer", "data_scientist"]
    }
  }`
}

type errorHandler func(http.ResponseWriter, *http.Request) error

func handleErrors(h errorHandler) http.HandlerFunc {
  return func(w http.ResponseWriter, r *http.Request) {
    err := h(w, r)
    if err == nil {
      return
    }
    
    w.WriteHeader(500)
    log.Println(err)
    w.Write([]byte(`{"error": "uh oh spaghettios"}`))
    return
  }
}

func (s *Server) handleRego(w http.ResponseWriter, r *http.Request) error {
  var input Request
  if err := json.NewDecoder(r.Body).Decode(&input); err != nil {
    return err
  }
  
  result, err := s.doTheRego(r.Context(), input)
  if err != nil {
    return err
  }
  
  return json.NewEncoder(w).Encode(result)
}

func (s *Server) doTheRego(ctx context.Context, input Request) (Response, error) {
  stringData := getData()
  d := json.NewDecoder(bytes.NewBufferString(stringData))
  d.UseNumber()
  var data map[string]interface{}
  if err := d.Decode(&data); err != nil {
    return Response{}, err
  }
  
  f := rego.Store(inmem.NewFromObject(data))
  f(s.Rego)
  
  eval, err := s.Rego.PrepareForEval(ctx)
  if err != nil {
    return Response{}, err
  }
  
  results, err := eval.Eval(ctx, rego.EvalInput(input))
  if len(results) == 0 {
    return Response{}, err
  }
  if err != nil {
    return Response{}, err
  }
  
  return handleResponse(results)
}

func handleResponse(results rego.ResultSet) (Response, error) {
  var result Response
  resp, err := json.Marshal(results[0].Bindings["x"])
  if err != nil {
    return result, err
  }
  
  if err := json.Unmarshal(resp, &result); err != nil {
    return result, err
  }
  
  return result, nil
}

func main() {
  s := Server{
    Rego: rego.New(
      rego.Load([]string{"applicants.rego"}, nil),
      rego.Query("x = data.applications"),
    ),
  }
  r := http.NewServeMux()
  r.Handle("/testing", handleErrors(s.handleRego))
  
  log.Fatal(http.ListenAndServe(":9090", r))
}

Conclusion

I’ve used this in a few different services that had fairly complex logic. What would have taken a few hundred lines of Go took around 50 lines of Rego. It’s not a silver bullet. You have to ensure the data passed in is valid, but Rego can natively leverage JSON schema, so that is a help. Rego also has a nice standard library which sometimes is worth it to call Rego for tasks that are simple in Rego but harder in Go.

Validating In Go#

Validating In Rego#

Calling Rego From Go#

Putting It All Together#

Conclusion#

Validating In Go

Validating In Rego

Calling Rego From Go

Putting It All Together

Conclusion