Record post-submission runtime evidence (#9)

Document the new Reddit runtime feedback, Pisama corpus provenance, licensing boundary, and maintainer-reported Codex for OSS submission status.
This commit is contained in:
Felipe Domingues 2026-07-24 07:30:10 -03:00 committed by GitHub
parent cb4d664e98
commit 4a4f103ddc
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 41 additions and 2 deletions

View file

@ -16,6 +16,26 @@ The v0.9 scope came from the public [original r/n8n launch thread](https://www.r
Captured on 2026-07-22: the thread had 3 votes and several substantive comments. Treat the comments as product evidence; do not present the vote count as broad adoption.
## Post-release runtime feedback — 2026-07-24
Reddit user [`Fit_Preference_1795`](https://www.reddit.com/r/n8n/comments/1v3is1w/comment/ozbyh0f/) publicly described the runtime counterpart to Vibeflow's static checks and offered to test anonymized failing workflows against Vibeflow. The report identified three failure classes:
- a `continueOnFail` Code node crash that leaves the execution marked successful;
- an error object flowing through an item's JSON as ordinary data;
- truncated AI output reaching downstream nodes despite valid JSON and a successful node status.
The commenter also noted that intentional token caps must be distinguished from accidental truncation to avoid false positives. These cases reinforce the product boundary: Vibeflow can detect deterministic fail-open configuration or require a structural completeness guard, but it cannot observe runtime output or prove that an alert is watched.
The linked [`n8n-nodes-pisama`](https://github.com/Pisama-AI/n8n-nodes-pisama) community node is MIT-licensed and forwards execution telemetry. Runtime detection is implemented by the separate, self-hostable [`pisama-n8n`](https://github.com/Pisama-AI/pisama-n8n) service under a fair-code license. Its public [July 2026 corpus campaign](https://github.com/Pisama-AI/pisama-n8n/blob/main/eval/campaigns/2026-07-guard-campaign.md) records 70 committed workflows, 67 real executions, and 19 observed-and-detected failures. The comment's 69-workflow report and the repository campaign are different snapshots and must not be combined into one metric.
Evidence status at capture time:
- verified: substantive public technical feedback and a public offer to provide anonymized false-positive evidence;
- not yet verified: receipt of the workflows, Vibeflow verdicts, false-positive count, or a shared interoperability contract;
- not claimed: partnership, external contribution, external user adoption, or a shipped Pisama integration.
The smallest useful follow-up is three anonymized cases: fail-open Code execution, error-as-data, and intentional versus accidental truncation. A new Vibeflow rule should be added only when paired unsafe/safe fixtures expose a deterministic static signal.
## Reddit — r/n8n
### Title