Connect
The planned connectors cover REST APIs, MQTT, webhooks, and databases. Each source keeps its connection and schema assumptions visible.
INGESTION PIPELINE
Make external data connections easier to inspect and operate.
A product concept for reusable source connectors, transformations, validation, replay, and observability across distributed data systems.
Product concept · Not yet for sale
One path, explicit checkpoints
The concept turns source-specific integration work into a sequence an operator can inspect.
Transform and validate
Rules would normalize records toward target formats such as NGSI-LD or JSON-LD, then apply deterministic validation before delivery.
Observe and recover
The design would retain failed records and their error context, so operators can correct a rule and reprocess only the affected scope.
A reviewable delivery flow
The product concept keeps source data, mapping decisions, validation, and human operations distinct.
-
01
Describe source and target
Provide a source schema, sample payload, and target constraints.
-
02
Draft and review mapping
Inspect transformation choices before applying any connection configuration.
-
03
Validate deterministically
Use explicit checks before transformation and delivery.
-
04
Review failures
Read the source evidence and decide the next operational action.
Illustrative planned workflow
Source: parking sensor payload
Target: NGSI-LD entity constraints
Result: mapping draft for human approval, then deterministic validation
Claude proposes; the operator approves
We want to reduce the time spent building field maps and reading long logs. In the planned workflow, Claude proposes a draft and the operator checks its evidence before applying a change.
Current: Claude Code development workflow
We use Claude Code to organize source requirements and design connector boundaries, transformation rules, and failure-case checks. People own architecture and deployment decisions.
Planned: mapping assistance
We plan to send source schemas, sanitized samples, and target constraints through a server to the Claude API. It would return field mappings and transformation drafts, leaving ambiguous units as questions for an operator to resolve and approve.
Planned: failure investigation
Sanitized failure logs and validation rules would be used to propose possible causes, supporting log references, and next checks. Operators would compare the original evidence and decide what action to take.
Questions
Can Claude change a pipeline configuration?
No. The planned role is a draft and explanation for a human to review. Deterministic validation and approved changes remain separate steps.
How would this be evaluated?
In an MVP, we plan to measure how much operators edit each draft, whether mapping errors decrease, and the cost per response.
Product conversation