Announcing Topaz 0.32!

May 16th, 2024

Omri Gazitt

Topaz

The latest version of Topaz is out! The changes in this release focus on the ergonomics of the Topaz CLI. We’ve added configuration management, additional subcommands, and cleaned up the organization of the command trees. Read on for more!

Named configurations

We received a fair amount of feedback that the templates feature we added in Topaz 0.31 is very useful, but users wanted to be able to install more than one template, and easily switch between them. With Topaz 0.32, we’ve reimagined configuration management to enable this.

In previous versions, Topaz only supported a single active configuration. With 0.32, you can now create and manage multiple configurations using the topaz config commands.

Creating configurations

The easiest way to create a configuration is to use the topaz templates install command. Every template will install into its own named configuration, named after the template by default.

Let’s install two templates - todo and gdrive:

% topaz templates install todo

Installing this template will completely reset your topaz configuration.
Do you want to continue? (y/N) y
>>> stopping topaz...
>>> stopping topaz "todo"...
>>> topaz is not running
>>> configure policy
certs directory: /Users/ogazitt/.local/share/topaz/certs

  FILE            ACTION
  gateway.crt     skipped, file already exists
  gateway-ca.crt  skipped, file already exists
  gateway.key     skipped, file already exists
  grpc.crt        skipped, file already exists
  grpc-ca.crt     skipped, file already exists
  grpc.key        skipped, file already exists
policy name: todo

Using configuration "todo"
>>> starting topaz "todo"...
036c8762497c8662804f2ab7c1d38a76c8d6c76aaf58a588ceaf098069f09e3e


WARNING: delete manifest resets all directory state, including relation and object data
>>> delete manifest>>> set manifest to /Users/ogazitt/.local/share/topaz/tmpl/todo/model/manifest.yaml>>> importing data from /Users/ogazitt/.local/share/topaz/tmpl/todo/data
        objects 20
      relations 25

You should see the console come up, displaying the Todo manifest:

Now let’s install the gdrive template:

% topaz templates install gdrive

Installing this template will completely reset your topaz configuration.
Do you want to continue? (y/N) y
>>> stopping topaz...
>>> stopping topaz "gdrive"...
>>> topaz is not running
>>> configure policy
certs directory: /Users/ogazitt/.local/share/topaz/certs

  FILE            ACTION
  gateway.crt     skipped, file already exists
  gateway-ca.crt  skipped, file already exists
  gateway.key     skipped, file already exists
  grpc.crt        skipped, file already exists
  grpc-ca.crt     skipped, file already exists
  grpc.key        skipped, file already exists
policy name: gdrive

Using configuration "gdrive"
>>> starting topaz "gdrive"...
4756387323c8f8c85b86ea17ca11fe25adfc50a8813c032e05a9b16b1ea6b407


WARNING: delete manifest resets all directory state, including relation and object data
>>> delete manifest>>> set manifest to /Users/ogazitt/.local/share/topaz/tmpl/gdrive/model/manifest.yaml>>> importing data from /Users/ogazitt/.local/share/topaz/tmpl/gdrive/data
        objects 31
      relations 46
0001 check            PASS  folder:root#owner@user:beth@the-smiths.com [true] (21.056875ms)
0002 check            PASS  folder:root#can_read@user:beth@the-smiths.com [true] (779.458µs)
0003 check            PASS  folder:root#can_write@user:beth@the-smiths.com [true] (749.667µs)
0004 check            PASS  folder:root#can_share@user:beth@the-smiths.com [true] (677.666µs)
0005 check            PASS  folder:root#can_read@user:rick@the-citadel.com [false] (535.917µs)
…

As you can see, this stopped Topaz, installed the new template, restarted Topaz, and opened the console to the gdrive manifest:

Listing configurations

We should now have two configurations: todo and gdrive, with gdrive being the active configuration.

% topaz config list

     NAME    CONFIG FILE
  *  gdrive  gdrive.yaml
     todo    todo.yaml

Changing the active configuration

We can switch to the todo configuration using topaz config use todo. Note that we first need to stop topaz before issuing this command. Once you do, Topaz will switch to use the config file, policy, data, and assertions for the new config. Also note that you’ll have to start topaz again once you’ve switched configs:

% topaz config use todo
topaz: error: topaz is already running, use 'topaz stop' to stop
% topaz stop
>>> stopping topaz "gdrive"...
% topaz config use todo

Using configuration "todo"
% topaz start
>>> starting topaz "todo"...
e4bbee15a07b9c107970014e9212aeae198fc3dc0b57847f9a3e11dc3688406f

% topaz console

You should now see the Todo manifest again:

Creating a new configuration from scratch

The topaz configure command in previous versions of Topaz has been renamed topaz config new. You can create a new configuration by specifying a policy image.

% topaz config new --help
Usage: topaz config new --force [flags]

create new configuration

Flags:
  -h, --help                         Show context-sensitive help.
  -N, --no-check                     disable local container status check ($TOPAZ_NO_CHECK)
  -L, --log                          log level

  -n, --name=CONFIG-NAME             config name
  -l, --local-policy-image=STRING    local policy image name
  -r, --resource=STRING              resource url
  -p, --stdout                       print to stdout
  -d, --edge-directory               enable edge directory
  -f, --force                        skip confirmation prompt

% topaz config new -n my-peoplefinder -r ghcr.io/aserto-policies/policy-peoplefinder-rebac:latest
>>> configure policy
certs directory: /Users/ogazitt/.local/share/topaz/certs

  FILE            ACTION
  gateway.crt     skipped, file already exists
  gateway-ca.crt  skipped, file already exists
  gateway.key     skipped, file already exists
  grpc.crt        skipped, file already exists
  grpc-ca.crt     skipped, file already exists
  grpc.key        skipped, file already exists
policy name: my-peoplefinder

We now have a new configuration that uses one of the Aserto peoplefinder policies. Let’s switch to it:

% topaz config list

     NAME             CONFIG FILE
  *  gdrive           gdrive.yaml
     my-peoplefinder  my-peoplefinder.yaml
     todo             todo.yaml
% topaz stop
>>> stopping topaz "gdrive"...
% topaz config use my-peoplefinder

Using configuration "my-peoplefinder"
% topaz start
>>> starting topaz "my-peoplefinder"...
658ab64079bf995762bf1e0a6e6226fab0763e6942a4e45a8c8ecc1e2c1ded5f

% topaz console

We will now see an empty manifest since we haven’t created a model.

But we can switch to the “Policy Details” tab and see that we’ve loaded the peoplefinder-rebac policy:

To round out the feature set, you can of course rename and delete configurations, but note that Topaz must not be running the configuration you’re trying to rename or delete.

Configuration and data locations

Topaz 0.32 follows the XDG base directory specification to determine the locations of config and data files. Topaz honors the XDG_CONFIG_HOME and XDG_DATA_HOME environment variables and places config and template files in $XDG_CONFIG_HOME/topaz, and data files in $XDG_DATA_HOME/topaz. If these aren't set, their defaults are:

MacOS and Linux: $HOME/.config/topaz and $HOME/.local/share/topaz
Windows: $HOME\AppData\Local

To find out where your config and data files are, you can use the new topaz config info set of commands:

% topaz config info
{
  "environment": {
    "home": "/Users/ogazitt",
    "xdg_config_home": "/Users/ogazitt/.config",
    "xdg_data_home": "/Users/ogazitt/.local/share"
  },
  "config": {
    "topaz_cfg_dir": "/Users/ogazitt/.config/topaz/cfg",
    "topaz_certs_dir": "/Users/ogazitt/.local/share/topaz/certs",
    "topaz_db_dir": "/Users/ogazitt/.local/share/topaz/db",
    "topaz_tmpl_dir": "/Users/ogazitt/.local/share/topaz/tmpl",
    "topaz_dir": "/Users/ogazitt/.config/topaz"
  },
  "runtime": {
    "active_configuration_name": "peoplefinder",
    "active_configuration_file": "/Users/ogazitt/.config/topaz/cfg/peoplefinder.yaml",
    "running_configuration_name": "peoplefinder",
    "running_configuration_file": "/Users/ogazitt/.config/topaz/cfg/peoplefinder.yaml",
    "running_container_name": "topaz-peoplefinder",
    "topaz_json": "/Users/ogazitt/.config/topaz/topaz.json"
  },
  "default": {
    "container_registry": "ghcr.io/aserto-dev",
    "container_image": "topaz",
    "container_tag": "0.32.5",
    "container_platform": "linux/arm64",
    "topaz_no_check": false
  },
  "directory": {
    "topaz_directory_svc": "localhost:9292",
    "topaz_directory_key": "",
    "topaz_directory_token": "",
    "topaz_insecure": false,
    "aserto_tenant_id": ""
  },
  "authorizer": {
    "topaz_authorizer_svc": "localhost:8282",
    "topaz_authorizer_key": "",
    "topaz_authorizer_token": "",
    "topaz_insecure": false,
    "aserto_tenant_id": ""
  }
}

If you'd like to obtain the value of a specific configuration value, you can obtain it using a JSON "dot-syntax", like this:

% topaz config info config.topaz_cfg_dir
"/Users/ogazitt/.config/topaz/cfg"

This can be very useful for writing scripts that require knowing the location of configuration information.

Topaz directory commands

We’ve moved the directory-oriented commands into their own subtree, topaz directory. The import, export, backup, restore commands are all now in this subtree, as are the topaz test commands that relate to directory assertions.

But the most exciting new features are the ability to execute sub-commands that get and set object, relation, and the manifest, as well as execute check and search commands.

% topaz ds --help
Usage: topaz directory (ds) <command> [flags]

directory commands

Commands:
  directory (ds) check      check permission
  directory (ds) search     search relation graph
  directory (ds) get        get object|relation|manifest
  directory (ds) set        set object|relation|manifest
  directory (ds) delete     delete object|relation|manifest
  directory (ds) list       list objects|relations
  directory (ds) import     import directory data
  directory (ds) export     export directory data
  directory (ds) backup     backup directory data
  directory (ds) restore    restore directory data
  directory (ds) test       execute directory assertions

Flags:
  -h, --help        Show context-sensitive help.
  -N, --no-check    disable local container status check ($TOPAZ_NO_CHECK)
  -L, --log         log level

Listing and getting objects and relations

Note that the examples below all use the peoplefinder template.

To list all the objects of a certain type:

% topaz directory list objects '{"object_type":"user"}' --insecure
{
  "results":  [
    {
      "type":  "user",
      "id":  "aaronp@acmecorp.com",
      "display_name":  "Aaron Painter",
      "properties":  {
        "connection_id":  "d0bc8d47-2c0b-11ee-b3f8-0328466d27be",
        "department":  "Strategy Consulting",
        "email":  "aaronp@acmecorp.com",
        "enabled":  true,
        "manager":  "christk@acmecorp.com",
        "phone":  "+1-212-555-8335",
        "picture":  "https://www.topaz.sh/assets/templates/acmecorp/img/Aaron%20Painter.jpg",
        "roles":  [
          "user",
          "acmecorp",
          "strategy-consulting",
          "viewer"
        ],
        "status":  "USER_STATUS_ACTIVE",
        "title":  "Strategy Consulting Manager"
      },
      "created_at":  "2024-04-25T04:10:56.095213004Z",
      "updated_at":  "2024-04-25T04:10:56.095213004Z",
      "etag":  "12359342880935013463"
    },
    {
      "type":  "user",
      "id":  "adamb@acmecorp.com",
      "display_name":  "Adam Barr",
      "properties":  {
        "connection_id":  "d0bc8d47-2c0b-11ee-b3f8-0328466d27be",
        "department":  "Operations",
        "email":  "adamb@acmecorp.com",
        "enabled":  true,
        "manager":  "danj@acmecorp.com",
        "phone":  "+1-206-555-5472",
        "picture":  "https://www.topaz.sh/assets/templates/acmecorp/img/Adam%20Barr.jpg",
        "roles":  [
          "user",
          "acmecorp",
          "operations",
          "admin"
        ],
        "status":  "USER_STATUS_ACTIVE",
        "title":  "General Manager of Professional Services"
      },
      "created_at":  "2024-04-25T04:10:56.093391254Z",
      "updated_at":  "2024-04-25T04:10:56.093391254Z",
      "etag":  "1551785326371979169"
    },
  ...
  }
}

To get a specific object by type and ID:

% topaz directory get object '{"object_type":"user", "object_id": "euang@acmecorp.com"}' --insecure
{
  "result":  {
    "type":  "user",
    "id":  "euang@acmecorp.com",
    "display_name":  "Euan Garden",
    "properties":  {
      "connection_id":  "d0bc8d47-2c0b-11ee-b3f8-0328466d27be",
      "department":  "Sales Engagement Management",
      "email":  "euang@acmecorp.com",
      "enabled":  true,
      "manager":  "aprils@acmecorp.com",
      "phone":  "+1-804-555-3383",
      "picture":  "https://www.topaz.sh/assets/templates/acmecorp/img/Euan%20Garden.jpg",
      "roles":  [
        "user",
        "acmecorp",
        "sales-engagement-management",
        "viewer"
      ],
      "status":  "USER_STATUS_ACTIVE",
      "title":  "Salesperson"
    },
    "created_at":  "2024-04-25T04:10:56.101510462Z",
    "updated_at":  "2024-04-25T04:10:56.101510462Z",
    "etag":  "3418643365887736106"
  },
  "relations":  [],
  "page":  {
    "next_token":  ""
  }
}

You can use the topaz directory get relation and topaz directory list relations commands to do the same for relations.

Setting and deleting objects and relations

To create a new object:

topaz ds set object '{"object": { "type": "user", "id": "newuser@acmecorp.com", "display_name": "New User" } }' --insecure
{
  "result":  {
    "type":  "user",
    "id":  "newuser@acmecorp.com",
    "display_name":  "New User",
    "properties":  {},
    "created_at":  "2024-05-20T01:09:01.062054627Z",
    "updated_at":  "2024-05-20T01:09:01.062054627Z",
    "etag":  "698080936132874497"
  }
}

Updating objects is done using an optimistic concurrency model. Note that the object contains an etag field, which must be used in a future set command if you want to overwrite the object's value.

To delete the object we just created:

% topaz ds delete object '{"object_type": "user", "object_id": "newuser@acmecorp.com" }' --insecure
{
  "result":  {}
}

The same set of commands can be used to set and delete relations.

Check and search

To check whether the user euang@acmecorp.com is a member of the viewer group, issue the following command:

% topaz ds check '{"object_type": "group", "object_id": "viewer", "relation": "member", "subject_type": "user", "subject_id": "euang@acmecorp.com" }' --insecure
{
  "check":  true,
  "trace":  []
}

To determine which groups the user euang@acmecorp.com is a member of, we can use a search request. The format is exactly the same as the check request, but you can omit the object_id field to return all the objects of type group that have a member relation to this user:

% topaz ds search '{"object_type": "group", "relation": "member", "subject_type": "user", "subject_id": "euang@acmecorp.com" }' --insecure
{
  "results":  [
    {
      "object_type":  "group",
      "object_id":  "user"
    },
    {
      "object_type":  "group",
      "object_id":  "viewer"
    },
    {
      "object_type":  "group",
      "object_id":  "acmecorp"
    },
    {
      "object_type":  "group",
      "object_id":  "sales-engagement-management"
    }
  ],
  "explanation":  null,
  "trace":  []
}

We can also search the other way. To determine which users are members of the editor group, issue the following search command, this time omitting the subject_id field to return all the subjects that fit the search:

% topaz ds search '{"object_type": "group", "object_id": "editor", "relation": "member", "subject_type": "user" }' --insecure
{
  "results":  [
    {
      "object_type":  "user",
      "object_id":  "arturol@acmecorp.com"
    },
    {
      "object_type":  "user",
      "object_id":  "dianet@acmecorp.com"
    },
    {
      "object_type":  "user",
      "object_id":  "kellyw@acmecorp.com"
    }
  ],
  "explanation":  null,
  "trace":  []
}

Topaz authorizer commands

Topaz has a new set of commands under topaz authorizer:

% topaz authorizer --help
Usage: topaz authorizer (az) <command> [flags]

authorizer commands

Commands:
  authorizer (az) eval             evaluate policy decision
  authorizer (az) query            execute query
  authorizer (az) decisiontree     get decision tree
  authorizer (az) get-policy       get policy
  authorizer (az) list-policies    list policies
  authorizer (az) test             execute authorizer assertions

Flags:
  -h, --help        Show context-sensitive help.
  -N, --no-check    disable local container status check ($TOPAZ_NO_CHECK)
  -L, --log         log level

Listing policy modules

To list the policies that are loaded into the authorizer, use the list-policies subcommand:

% topaz az list-policies '{ "field_mask": "id" }' --insecure
{
  "result":  [
    {
      "id":  "peoplefinder/github/workspace/content/src/policies/get.rego"
    },
    {
      "id":  "peoplefinder/github/workspace/content/src/policies/post.rego"
    },
    {
      "id":  "peoplefinder/github/workspace/content/src/policies/__id/delete.rego"
    },
    {
      "id":  "peoplefinder/github/workspace/content/src/policies/__id/get.rego"
    },
    {
      "id":  "peoplefinder/github/workspace/content/src/policies/__id/post.rego"
    },
    {
      "id":  "peoplefinder/github/workspace/content/src/policies/__id/put.rego"
    }
  ]
}

To get the raw policy output for one of the modules, use the get-policy subcommand:

% topaz az get-policy '{ "field_mask": "raw", "id": "peoplefinder/github/workspace/content/src/policies/get.rego" }' --insecure
{
  "result":  {
    "raw":  "package peoplefinder.GET.api.users\n\n# Policy for retrieving the list of employees in PeopleFinder\n#\n# Retrieving the list of employees is Allowed if:\n# - True\n#\n# The list of employees is Visible if:\n# - True\n#\n# The list of employees is Enabled if:\n# - True\n\ndefault allowed = true\n\ndefault visible = true\n\ndefault enabled = true\n"
  }
}

Evaluating decisions and queries

To evaluate a decision using a policy path, use the following command:

% topaz authorizer eval '{
  "identity_context": {
    "type": "IDENTITY_TYPE_SUB",
    "identity": "euang@acmecorp.com"
  },
  "policy_context": {
    "decisions": [
      "allowed"
    ],
    "path": "peoplefinder.GET.api.users"
  }
}' --insecure
{
  "decisions":  [
    {
      "decision":  "allowed",
      "is":  true
    }
  ]
}

You can similarly issue a decision-tree or query subcommand. For example, for a query:

% topaz authorizer query '{
  "identity_context": {
    "type": "IDENTITY_TYPE_SUB",
    "identity": "euang@acmecorp.com"
  },
  "query": "x = data.peoplefinder.GET.api.users.__id"
}' --insecure
{
  "response":  {
    "result":  [
      {
        "bindings":  {
          "x":  {
            "allowed":  true,
            "enabled":  true,
            "visible":  true
          }
        },
        "expressions":  [
          {
            "location":  {
              "col":  1,
              "row":  1
            },
            "text":  "x = data.peoplefinder.GET.api.users.__id",
            "value":  true
          }
        ]
      }
    ]
  },
  "metrics":  {},
  "trace":  [],
  "trace_summary":  []
}

Topaz test

The topaz test command featured in previous versions of Topaz has been re-factored into two - topaz directory test and topaz authorizer test. The former set of commands can be used to create a template file that describes directory check calls, and to execute those check calls against the Topaz directory. The latter does the same thing, but for authorizer is calls. Since these execute different types of tests, we chose to separate them into two distinct commands.

That's all for now!

As you can see, starting with Topaz 0.32, you can do a lot more using the Topaz CLI. Please let us know if you have any feedback in our community slack, and enjoy the new release!