Text column edges

Where coding agents go wrong, and how Railspike helps

You ask an agent to fix a CSV export. It rewrites the export helper, adds a dependency you ruled out, and says "done" after testing the helper in isolation. You open the app and the Download button still fails. That's several different problems in one diff.

Tang and colleagues studied 20,574 real-world coding-agent sessions. Among the episodes where developers visibly pushed back, the most frequent symptoms were ignored constraints (38%), misunderstood intent (27%), and inaccurate status reports (23%). In Stack Overflow's 2025 survey, 66% of respondents to the AI-frustrations question reported dealing with almost-correct answers, and 45% reported that debugging AI-generated code took more time. Coding-agent study, Table 3 · Stack Overflow survey

Railspike is a set of Markdown instructions for your existing coding agent. Its staged workflow runs through optional assessment, a feature description, a development plan, and implementation. You can edit the planning files in the repo and explicitly approve each step before the agent proceeds.

"It ignored a constraint I gave it"

Railspike asks the agent to read the repo's instructions and carry the approved boundaries into its plan. If the rule is "use the existing CSV writer and add no dependency," you can see it in the plan before code changes. If the agent later finds that completing the task requires changing the approved scope, it has to bring that change back for approval. The diff review checks the work against the same boundaries.

"It built the wrong thing"

The feature description makes the intended behavior concrete before you approve it. For an export, that might mean downloading the currently filtered rows through the existing button, with column order preserved. If the request has several plausible interpretations, the optional assessment gives you a place to compare them or combine parts of proposed approaches while the decision is still cheap to change.

"It said it was done, but I still can't use it"

The development plan records the completion checks up front. During implementation the agent records what it ran, what passed, and what is still blocked or pending. A unit test for CSV serialization answers a different question from downloading the correct file through the app. Railspike asks for verification through the normal entry point and keeps implementation, merge, and release status distinct.

"The code is almost right, and I'm doing the debugging"

The plan breaks staged work into pieces with their own checks. Implementation happens on a feature branch, with stage-scoped commits and a summary of the changes and results. When a later check exposes a regression, you have smaller diffs and a record of what was exercised. The shared implementation instructions also call for reproducing reported failures in the relevant environment and checking retry or recovery where the task requires it.

"It guessed how the repo works"

Railspike asks the agent to inspect the actual call paths, find the existing components and contracts, and record what it will reuse or extend. For the export, that could mean finding where the UI applies filters and where the server already serializes rows. The plan gives you a chance to catch a bad assumption with source paths in front of you.

"I asked for a fix and got a refactor"

The approved outcome and file boundaries give you something concrete to compare with the diff. The agent checks each staged commit for unrelated changes. If it spots an opportunity to replace the whole export subsystem, that can be discussed separately while the original fix stays reviewable.

"I opened a new session and had to explain everything again"

The approved requirements, plan, and implementation summary live in the repo. On resumption, Railspike asks the agent to verify the approved files and commits and recover the current stage. The next session can pick up from the decision you actually made and the checks that still need to run.

"The agent ran the wrong command or checked the wrong environment"

Railspike can surface this through the plan and verification record. Name the target environment, record the command and its result, and require evidence from the environment where the problem occurs. The workflow still depends on the agent and reviewer checking that the command targeted the right place.

The tradeoff is time spent reviewing plans. Railspike also has a one-shot mode for a bounded task you authorize upfront. It keeps scope, decisions, and verification in one evolving record and skips the intermediate planning approvals. Both modes rely on the agent following written instructions. Tool permissions and CI checks are separate controls. Railspike workflow and modes · Shared implementation standards

Which of these costs you the most time with coding agents? If you've tried Railspike, where did it save a round of rework, and where did it add one?

by guest on · 1 reply

Reply
#

by guest (unapproved) on

Coding agents rarely fail by writing bad code. They fail by not listening.

Tang et al. looked at 20,574 real coding-agent sessions across 1,639 repositories and studied every moment a developer pushed back. Broken code came fourth, at 18%. The top three failures were:

  • Ignoring a stated constraint (38%). Terminal agents did this much more often than IDE agents, 49% vs. 32%.

  • Misreading what the developer wanted (27%).

  • Reporting work as done when it wasn't (23%).

Two other findings stood out:

  • Agents almost never caught their own mistakes. Among episodes with a visible fix, only 3% were self-corrected, and 91% needed the developer to step in.

  • Over the study period, overall misalignment went down, but constraint violations and false "done" reports made up a growing share. Better models are fixing the code more than the listening.

That matches Stack Overflow's 2025 survey. There, 66% of respondents said AI answers are often "almost right, but not quite," and 45% said debugging AI-generated code takes longer.

What helps, with or without a tool

None of this needs special software. These habits each target one of the failures above:

  1. Write the constraints where the agent has to repeat them back. "No new dependencies, use the existing CSV writer" should appear in a plan you read before any code changes, not just in a chat message from 40 turns ago.

  2. Define "done" before work starts. "The serializer unit test passes" and "Download gives me the filtered rows in the right column order" are different claims. Ask for evidence of the second one.

  3. Make the agent cite the code before it plans. "I'll reuse exportRows() in server/export.ts" is easy to check. "I'll update the export logic" isn't.

  4. Keep work small and on a branch. Commit once per step, each commit with its own check. When something breaks later, you know which small diff caused it.

  5. Store decisions in the repo, not the chat. A new session should read the plan and pick up where the last one stopped, so you don't have to explain everything again.

  6. Keep real enforcement outside the prompt. Written rules are requests. Tool permissions, branch protection, and CI are what actually stop an agent.

Where Railspike fits

Railspike packages habits 1-5 as Markdown instructions for the agent you already use. Work goes through stages: an optional assessment, a feature description, a plan, and implementation. Each stage leaves a file in the repo that you can edit and approve. If the agent needs to go beyond the approved scope, it has to ask first. It also tracks "implemented," "merged," and "released" as separate states.

Here's what it doesn't do:

  • It doesn't cover habit 6. It's still a prompt, and the agent can ignore it.

  • It costs review time. For small tasks, a one-shot mode skips the approvals between stages.

  • It can't verify where commands ran. It logs what ran and where, but a person still has to confirm it was the right environment.

Your experience

Which of these failures costs you the most time? And if you've already solved one of them, how? I'd like to hear approaches that work without any tool.

Reply
#
…
Options