Skip to content

Native Config Post-Processing (NCPP)

This feature allows the use of all the underlying Envoy capabilities even if they are not (yet) configurable via the Core WAAP custom resource.

Introduction

The Core WAAP Operator translates the Core WAAP custom resource (CR) to native configuration of the Envoy proxy. The resulting native configuration is also in the form of YAML files.

The "native configuration post-processing" feature allows to adapt the generated Envoy configuration before it is deployed, using JavaScript. This allows to freely modify the effective configuration by adding, changing or removing arbitrary settings.

A very basic example to make the concept clear: Below the beginning of the Envoy config file envoy.yaml as it is currently generated by default:

node:
  id: "core.waap.id"
  cluster: "core.waap.cluster"
# ... more generated settings ...

Suppose that for some reason you wanted to change the id to core.waap.id.modified-by-post-processing (not a practically useful example, just to explain the concept, more realistic examples further below). Then you could configure the following in the nativeConfigPostProcessing section of the CR:

# ... other settings (routes, authentications, etc.) ...

nativeConfigPostProcessing:
- "envoy.node.id = 'core.waap.id.modified-by-post-processing'"

The configured JavaScript gets a variable envoy at runtime that contains the parsed YAML content of envoy.yaml. Hence envoy.node.id selects the node ID in the YAML tree structure and setting it to a different value overwrites the previous value. The operator takes care of translating the JavaScript variables back to YAML files, i.e. in the end also generates an envoy.yaml that is then deployed:

node:
  id: "core.waap.id.modified-by-post-processing"
  cluster: "core.waap.cluster"
# ... more generated settings ...

Currently, three native Enovy YAML files and corresponding JavaScript variables are generated:

  • envoy.yaml (envoy): Root Envoy configuration with some basic settings, also references the two files below.
  • lds.yaml (lds): Envoy listener configuration ("frontend"), incl. filter chains.
  • cds.yaml (cds): Envoy cluster configuration ("backends").

In the following a couple of real examples that were needed and worked at the time. Note that they may not be needed or not work any more by the time you are reading this, but they should be specific enough to show the possibilities.

After that, some general comments about use cases, best practices and potential future improvements.

Examples

Example: File upload in large chunks

By default, Envoy has a limit on uploads to chunks of maximally 1 MB when in combination with the Coraza filter (CRS enabled). This default behavior can be changed by explicitly setting perConnectionBufferLimitBytes on the listener, in the example below set to 10 MB in lds.yaml:

resources:
- '@type': "type.googleapis.com/envoy.config.listener.v3.Listener"
  name: "core.waap.listener"
  address:
    socketAddress:
      address: "0.0.0.0"
      portValue: 8080
  filterChains:
  # ... more config omitted ...
  perConnectionBufferLimitBytes: 10485760

As of writing this, there is only a single listener resource in the lds.yaml, hence the native config post-processing can be written like this:

nativeConfigPostProcessing:
- "lds.resources[0].perConnectionBufferLimitBytes = 10485760"

General Comments

Use cases and best practices

Quick workarounds for issues in production

This allows to fix at least some imminent problems without the need to create a bugfix release immediately.

This can be especially useful in cases where a fix cannot be easily tested outside production, reducing turnaround times and thus a quicker fix in production.

Obviously, in most cases the idea is to afterward include a corresponding fix into the next regular Core WAAP release.

Quick and flexible integrations of something new

A PoC is maybe the most general example for this use case, where integration issues can be solved ad-hoc whenever they occur without immediate need for a release, again also reducing turnaround times in cases where the feature cannot be easily/immediately tested outside the environment of the customer.

Depending on the nature of the integration, it may later make sense to include new features into the Core WAAP that cover the new integrations, namely if the integrations are of a kind that is generally useful, or are the better choice for other reasons.

Side remark: Note that this use case is often also interesting internally at USP because it allows to explore new settings sometimes more quickly than operating directly with native Envoy config.

Tuning / debugging existing installations

Use features that are present in the underlying Envoy but not (yet) configurable via the Core WAAP custom resource.

Future Improvements

We are evaluating various ways to make using this feature generally more easy and also more robust regarding new Core WAAP releases.

Note that the generated native Envoy config is subject to change:

Its structure may change, due to changes in the Core WAAP Operator or also due to changes in the underlying Envoy itself, hence it is not an API that is in any way guaranteed to remain stable, while in practice many things will still be quite stable most of the time.

As an example of potential future enhancements, JavaScript functions may be offered to get/set the Core Rule Set (CRS) settings of the Coraza filter more easily. And/or similar JavaScript functions that make it both more simple and potentially also more robust to get/set some specific or general items in the native Envoy config.