CWE-1434
Insecure Setting of Generative AI/ML Model Inference Parameters
Description
The product has a component that relies on a generative AI/ML model configured with inference parameters that produce an unacceptably high rate of erroneous or unexpected outputs.
{"xhtml:p":["Generative AI/ML models, such as those used for text\n\t\tgeneration, image synthesis, and other creative tasks, rely on\n\t\tinference parameters that control model behavior, such as\n\t\ttemperature, Top P, and Top K. These parameters affect the\n\t\tmodel's internal decision-making processes, learning rate, and\n\t\tprobability distributions. Incorrect settings can lead to\n\t\tunusual behavior such as text \"hallucinations,\" unrealistic\n\t\timages, or failure to converge during training. The impact of\n\t\tsuch misconfigurations can compromise the integrity of the\n\t\tapplication. If the results are used in security-critical\n\t\toperations or decisions, then this could violate the intended\n\t\tsecurity policy, i.e., introduce a vulnerability."]}
Parent Weaknesses (ChildOf)
Related Weaknesses
Common Consequences
Scope
Impact
Varies by Context, Unexpected State
Scope
Impact
Alter Execution Logic, Unexpected State, Varies by Context
Potential Mitigations
Develop and adhere to robust parameter tuning processes that include extensive testing and validation.
Implement feedback mechanisms to continuously assess and adjust model performance.
Provide comprehensive documentation and guidelines for parameter settings to ensure consistent and accurate model behavior.
Applicable Platforms
Security Training
Train your team to recognize and prevent security threats with our comprehensive security awareness program.
Start TrainingVulnerability Scanning
Discover vulnerabilities in your applications and infrastructure before attackers do.
Scan Now