How ScholarSignal evaluates your idea
Every evaluation follows the same pipeline — here is exactly what runs, on which model, against which rubric, and where it stops being reliable.
Step by step
- Live literature search. Your idea is turned into construct-aware OpenAlex queries — the theories, constructs and effects it actually invokes, not its surface wording. If you named a target journal, that journal is searched separately. An LLM relevance judge then keeps only works from the same scholarly conversation, plus field anchors: papers your reviewers would consider the home literature even when the constructs differ. Confirmed-relevant works are expanded along the citation graph.
- Four gates. Contribution viability, claim–design alignment, journal fit and desk-reject risk, judged against the retrieved works. The whole evaluation runs three times independently and the run carrying the median verdict wins — a single unlucky sample cannot decide your GO / REFINE / PIVOT / RETIRE.
- Verdict and venues. The verdict, the closest existing works, the closest authors and the journal recommendations are all grounded in that retrieved neighbourhood — venues are named because this literature publishes there, not because they sound plausible.
The model
The evaluation runs on OpenAI GPT-5.6 Terra via OpenRouter. Embeddings use OpenAI text-embedding-3-small. The weekly radar adds one more model: its ranking step — which reads your project title, description and profile — runs on Google Gemini 3.6 Flash, also via OpenRouter and under the same routing restriction. Every step is routed only to providers OpenRouter classifies as not collecting or training on submitted text, and whenever the routing layer reports it, the provider that served an evaluation is named in the report footer.
The rubric
The rubric is distilled from roughly 15 published editorials and editor essays on what survives desk review — the public record of what editors say they reject and why. No private review files or rejection letters are involved, and nothing is fine-tuned: the rubric lives in the prompt, not in model weights.
Limits
- The rubric has been checked against a small set of papers with known outcomes. It has not yet been validated against submission outcomes at scale.
- The literature search is broad but not exhaustive — OpenAlex coverage and query construction both have gaps, so a missed neighbour is always possible.
- A verdict is a structured second opinion, not a prediction of what any editor will do with your paper.
Your data
Your idea is unpublished work and is treated that way: never shared, never used to train AI models, used only to produce your evaluation and match opportunities for your weekly radar. It is stored in the EU and deleted on request.
The details — what is stored, for how long, which providers see the text, and your GDPR rights — are in the full privacy note.