Skip to main content

Http

An HTTP Source in Flowlyze reads and acquires data from remote endpoints over HTTP/HTTPS, typically exposed by web services, REST APIs, or microservices.
Data is processed as JSON, the standard for structured inter-system exchange.

Advanced HTTP Source configurations let you flexibly control many aspects of integration.

Incremental reading (Delta Reading)​

Configure the HTTP source to perform incremental reads, i.e., only new or changed data since the last run.
This uses Flowlyze’s memory and reduces transferred data volume.

Payload parsing​

Define parsing and transformation rules for incoming JSON using JSONPath expressions to identify fields, extract portions of data, or reshape the message for the downstream flow.

Supported authentication modes​

OAuth2 (Bearer Token)​

What it does: obtains an access token from an Authorization Server and sends it as Authorization: Bearer <token>.

When to use: modern enterprise/public APIs (OpenAPI), strong security, token expiry/rotation, scopes/permissions.

Typical Flowlyze config

  • Grant type: commonly Client Credentials for server-to-server (or Authorization Code for interactive users)
  • Token URL: OAuth2 endpoint (e.g., https://auth.example.com/oauth/token)
  • Client ID / Client Secret
  • Scope (optional)
  • Outbound header: Authorization: Bearer {{access_token}} (inserted automatically)

Custom JWT Bearer​

What it does: builds a signed JWT (typically RS256) with agreed claims and either sends it directly as Bearer or exchanges it for an access token at a custom endpoint.

When to use: proprietary APIs requiring a signed JWT instead of a standard OAuth token, or a custom “JWT → access token” flow.

Typical Flowlyze config

  • Algorithm: RS256/ES256/HS256 (usually RS256)

  • Private key / Key ID (kid)

  • Claims: iss, sub, aud, iat, exp, plus custom claims

  • Token emission:

    • Direct Bearer: send JWT as Authorization: Bearer <jwt>
    • Exchange: send JWT to an endpoint to obtain an access token, then use as Bearer

Final header example (direct bearer)

Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...

Basic Auth​

What it does: sends username:password Base64-encoded in the Authorization header.

When to use: legacy/internal services exposing Basic Auth over HTTPS.

Typical Flowlyze config

  • Username
  • Password
  • Outbound header (automatic)

Note: Always use HTTPS; otherwise Basic Auth credentials are exposed.

API Key​

What it does: sends a static key as header or query string.

When to use: simple/public services where OAuth is not required.

Typical Flowlyze config

  • Key value
  • Location: Header (e.g., x-api-key: <key>) or Query (e.g., ?apikey=<key>)

Complete example​

https://api.example.com/data?filter=update_date>{{last_update}}

The {last_update} placeholder is set by Flowlyze with the last stored incremental value (e.g., max update_date).

If the API returns this structure:

{
  "data": [{}, {}, {}]
}

Use data as the selector for records to ingest.