Full Report
LLMs are bad at generating passwords: There are strong noticeable patterns among these 50 passwords that can be seen easily: All of the passwords start with a letter, usually uppercase G, almost always followed by the digit 7. Character choices are highly uneven for example, L , 9, m, 2, $ and # appeared in all 50 passwords, but 5 and @ only appeared in one password each, and most of the letters in the alphabet never appeared at all. There are no repeating characters within any password. Probabilistically, this would be very unlikely if the passwords were truly random but Claude preferred to avoid repeating characters, possibly because it “looks like it’s less random”. ...
Analysis Summary
# Research: LLMs Generate Predictable Passwords (Analysis based on summary data)
## Metadata
- Authors: Referenced via a link to Irregular, implying external research was synthesized/discussed.
- Institution: Unspecified (The content is a blog post summarizing external research).
- Publication: Schneier on Security (Blog)
- Date: February 26, 2026
## Abstract
This analysis summarizes findings indicating that Large Language Models (LLMs), specifically Claude, generate low-entropy, highly predictable passwords. The generated password sets exhibit strong, non-random patterns in structure, character frequency, and repetition avoidance, making them significantly easier to compromise than truly random or human-generated complex passwords.
## Research Objective
The primary objective is to investigate the quality and randomness of passwords generated by Large Language Models (LLMs) when tasked with creating passwords. The implied question is: Do LLMs produce cryptographically acceptable, unpredictable secret strings?
## Methodology
### Approach
The research involved prompting an LLM (likely Claude) to generate a set of passwords (specifically 50 examples were sampled) and subsequently analyzing the structural characteristics and statistical distributions of those outputs.
### Dataset/Environment
The test set consisted of 50 passwords generated by the LLM Claude. Analysis focused on structural prefixes, character usage frequencies, and character repetition constraints within the generated set.
### Tools & Technologies
Not explicitly detailed, but involved text generation via an LLM and subsequent statistical analysis of the textual output.
## Key Findings
### Primary Results
1. **Strong Structural Bias:** Generated passwords exhibited highly consistent starting patterns, frequently beginning with an uppercase 'G' followed almost immediately by the digit '7' ($\text{G}7\dots$).
2. **Severely Skewed Character Distribution:** Character choices were highly non-uniform. A small subset of characters (e.g., L, 9, m, 2, \$, \#) appeared across all 50 passwords, while many other potential characters (e.g., 5, @, and most alphabet letters) were rarely or never selected.
3. **Forced Non-Repetition:** The model actively avoided repeating characters within any single generated password, suggesting a heuristic that perceived repeated characters as "less random."
4. **High Password Collisions:** Despite generating 50 attempts, only 30 unique passwords were produced. The most common password occurred 18 times ($36\%$ of the sample), indicating a high rate of identical outputs.
### Supporting Evidence
- The single most common password represented $36\%$ of the sample, which is vastly higher than the expected probability for a truly random password of comparable length/complexity ($2^{-100}$).
- The prevalence of the $\text{G}7$ prefix and the complete avoidance of the asterisk ($\ast$) suggest the LLM's internal training or formatting constraints (like Markdown interpretation) influence output character selection.
### Novel Contributions
The research empirically demonstrates that LLMs, when generating security artifacts like passwords, default to human-understandable patterns and heuristics rather than true cryptographic randomness, exposing systemic weaknesses in their generative capabilities for high-security tasks.
## Technical Details
The LLM's apparent internal constraint to avoid repeating characters seems to be driven by a learned association that repetition reduces perceived randomness, a concept that is inversely correlated with true entropy in password generation. Furthermore, the avoidance of the `*` symbol suggests artifacts from the training data's formatting language (Markdown) can leak into the generated strings.
## Practical Implications
### For Security Practitioners
Security teams relying on autonomous AI agents for account setup, provisioning, or token generation must recognize that LLM-generated credentials are not strong secrets and will be trivial for attackers using dictionary or pattern-based attacks derived from these observed LLM tendencies.
### For Defenders
Defenders should implement robust monitoring for credentials exhibiting the identified patterns (e.g., starting with specific common prefixes like G7, overly uniform structure, or lack of repeats) as these highly suggest LLM generation and significantly lower the assumed security strength.
### For Researchers
This finding highlights the need to rigorously test other security-critical outputs from LLMs (e.g., keys, initialization vectors, or obfuscation techniques) to ensure they are not governed by predictable, human-mimicking heuristics learned during training.
## Limitations
The studied set was small (50 passwords) and focused primarily on one model (Claude). Results may vary across different LLMs or different prompting strategies. The exact target password length and complexity requirements imposed on the LLM were not specified in the summary.
## Comparison to Prior Work
This work builds upon general criticisms of LLM stochasticity by focusing specifically on the security failure mode of password generation, contrasting existing password generation standards (which mandate maximal entropy and uniform distribution) with observed LLM behavior.
## Future Work
Further work should involve testing multiple LLMs against standardized password strength metrics, analyzing the effect of varying prompt complexity on the resulting bias, and developing mitigation techniques for autonomous agents during credential creation.
## References
- [External publication detailing the password results (Irregular)](https://www.irregular.com/publications/vibe-password-generation)
- Related news articles citing the findings.