genai-toolbox

Security Findings (292)

• CriticalAS-006

  💻Arbitrary Code Execution ×7

  heuristic signal

  tool name or description implies arbitrary script/code execution (evaluate_script, execute javascript, etc.)

  minify_jspr_gatekeepersimilarity_scorelist_local_testsrun_eval_contractgenerate_eval_yamlget_testing_guidelines

  Fix: This tool can execute arbitrary code or shell commands on the host system. Remove it unless strictly required. If kept: (1) restrict access to trusted users/agents only, (2) require human approval before each invocation (Claude Desktop: set approval_required: true; other clients: enable equivalent confirmation), (3) use the most restrictive sandbox or read-only mode available, and (4) never expose this tool to untrusted input sources.

• HighAS-003

  🔀Scope Mismatch ×2

  tool name "list_local_tests" implies read-only operation but declares exec permission

  list_local_tests

  tool name "get_testing_guidelines" implies read-only operation but declares exec permission

  get_testing_guidelines

  Fix: Ensure tool names, descriptions, and permission declarations are internally consistent. Use explicit naming conventions that fully reflect actual capabilities.

• LowAS-011

  ℹ️Missing Rate-Limit / Timeout ×43

  tool performs network or execution operations but declares no rate-limit, timeout, or retry configuration

  base64_encodeurl_encodeurl_decodegenerate_slugfetch_veille_feedmcp_server_evaluatejson_to_csvvalidate_urlcron_validatorembedding_similaritylatency_benchmarkvector_similarityhttp_status_lookupparse_http_headersgenerate_curlcount_code_linesextract_linksgenerate_hmacpr_gatekeepersecurity_headers_checkssl_certificate_checkcors_testcors_checkerwebhook_endpoint_createwebhook_endpoint_requestscookie_security_auditweb_security_auditsecret_scansimilarity_scorelist_local_testsrun_eval_contractrun_vlm_test_suitemultimodal_eval_guiderun_vlm_test_suite_batchgenerate_eval_yamlget_testing_guidelinesopenapi_validatepost_jira_commentcreate_confluence_pagefetch_jira_issuesearch_jira_issuesjira_to_test_suitefetch_confluence_page

  Fix: Declare explicit rate-limit, timeout, and retry configuration for all network and execution tools. Implement exponential back-off and surface resource state to the calling agent.

• LowAS-002

  ⚠️Excessive Permissions

  multimodal_eval_guide:input schema exposes 16 properties (threshold: 10)

  Fix: Tool requests broad permissions (exec/fs/network). Validate input parameters using Enums where possible, and restrict file system operations to explicit allowed directories.

• InfoAS-014

  ℹ️Dependency Inventory Unavailable ×146

  Tool did not expose metadata.dependencies or repo_url, so supply-chain coverage is limited.

  format_jsongenerate_uuidhash_textcount_tokensbase64_encodebase64_decodeurl_encodeurl_decodegenerate_slugvalidate_emailminify_jsdecode_jwttext_statsgenerate_passwordparse_csvcolor_convertregex_testlorem_ipsumtimestamp_convertdiff_texttruncate_to_tokenssplit_chunksextract_json_from_textstrip_markdownestimate_llm_costescape_htmlunescape_htmlfetch_veille_feedscore_geo_signalsextract_json_pathgenerate_json_ldanalyze_diff_bugsgenerate_test_casesrun_pr_gate_pipelinevalidate_mcp_responsellm_output_validatorcompare_responsesprompt_test_suitemcp_server_health_checkmcp_server_evaluatejson_schema_validateflatten_jsonxml_to_jsonredact_piimock_from_schematransform_json_arrayjson_to_csvcase_convertsort_linesnumber_base_convertvalidate_urlcheck_contrast_ratiohtml_to_markdowncron_parsecron_validatorcalculate_readabilitynormalize_whitespaceembedding_similarityllm_format_checkhallucination_checkprompt_injection_scantoken_budget_calculatorconsistency_checkllm_json_schema_checklatency_benchmarkresponse_quality_scorerag_relevance_ranktoxicity_scanguardrail_testfunction_call_validateconversation_analyzemcp_schema_lintcot_analyzerab_test_reportcontext_window_checkvector_similaritynormalize_vectorvector_quantizevector_statsbm25_scorebuild_rag_promptprompt_template_fillfew_shot_formattersystem_prompt_buildermodel_infocompare_modelshttp_status_lookupparse_http_headersgenerate_curlextract_todosdetect_secretscount_code_lineslint_commit_messageword_frequencyextract_linkslevenshtein_distancejson_diffmerge_jsonjson_to_yamlgenerate_hmacformat_bytesdetect_languagepr_gatekeeperlist_llm_modelsllm_generatererank_evaluateshield_analyzesecurity_headers_checkssl_certificate_checkcors_testcors_checkerwebhook_endpoint_createwebhook_endpoint_requestscookie_security_auditweb_security_auditsecret_scansimilarity_scoreneedle_haystack_generateoptimize_prompt_tokensbias_detectllm_fit_finderfind_toollist_local_testsrun_eval_contractgenerate_html_reportrun_vlm_test_suitemultimodal_eval_guiderun_vlm_test_suite_batchgenerate_eval_yamlvalidate_agent_trajectoryrun_semantic_testsget_testing_guidelinestest_skillidentify_calleryaml_to_jsonenv_parsejson_schema_generateformat_tableopenapi_validatepost_jira_commentcreate_confluence_pagefetch_jira_issuesearch_jira_issuesjira_to_test_suitefix_gherkinfetch_confluence_page

  Fix: Review and remediate the identified issue.

• InfoAS-002

  ⚠️Excessive Permissions ×58

  declared capabilities: database access

  generate_uuidunescape_htmlrag_relevance_rankbm25_scorebuild_rag_promptrerank_evaluate

  declared capabilities: network access

  base64_encodefetch_veille_feedmcp_server_evaluatelatency_benchmarkhttp_status_lookupextract_linkssecurity_headers_checkssl_certificate_checkcors_testcookie_security_auditweb_security_auditrun_vlm_test_suitemultimodal_eval_guiderun_vlm_test_suite_batchopenapi_validatepost_jira_commentfetch_jira_issuejira_to_test_suitefetch_confluence_page

  declared capabilities: filesystem access

  base64_decodestrip_markdownextract_json_pathtransform_json_arraysort_linesnormalize_whitespaceyaml_to_jsonenv_parse

  declared capabilities: network access, database access

  url_encodeurl_decodevalidate_urlsecret_scansearch_jira_issues

  declared capabilities: network access, filesystem access

  generate_slugcreate_confluence_page

  declared capabilities: HTTP requests

  json_to_csvembedding_similarityvector_similaritygenerate_hmac

  declared capabilities: code/command execution

  cron_validatorcount_code_linespr_gatekeepersimilarity_scoregenerate_eval_yaml

  declared capabilities: network access, HTTP requests

  parse_http_headerscors_checker

  declared capabilities: network access, filesystem access, HTTP requests

  generate_curlwebhook_endpoint_create

  declared capabilities: filesystem access, database access

  detect_secrets

  declared capabilities: network access, database access, HTTP requests

  webhook_endpoint_requests

  declared capabilities: code/command execution, filesystem access

  list_local_testsrun_eval_contract

  declared capabilities: code/command execution, database access

  get_testing_guidelines

  Fix: Tool requests broad permissions (exec/fs/network). Validate input parameters using Enums where possible, and restrict file system operations to explicit allowed directories.

• InfoAS-010

  🔑Insecure Secret Handling ×32

  input parameter "token" accepts a credential (informational; not evidence of insecure handling)

  decode_jwt

  input parameter "max_tokens" accepts a credential (informational; not evidence of insecure handling)

  truncate_to_tokensprompt_test_suitellm_generatejira_to_test_suite

  input parameter "chunk_tokens" accepts a credential (informational; not evidence of insecure handling)

  split_chunks

  input parameter "input_tokens" accepts a credential (informational; not evidence of insecure handling)

  estimate_llm_cost

  input parameter "output_tokens" accepts a credential (informational; not evidence of insecure handling)

  estimate_llm_cost

  input parameter "context_tokens" accepts a credential (informational; not evidence of insecure handling)

  token_budget_calculator

  input parameter "user_input_tokens" accepts a credential (informational; not evidence of insecure handling)

  token_budget_calculator

  input parameter "system_prompt_tokens" accepts a credential (informational; not evidence of insecure handling)

  token_budget_calculator

  input parameter "expected_output_tokens" accepts a credential (informational; not evidence of insecure handling)

  token_budget_calculator

  input parameter "max_output_tokens" accepts a credential (informational; not evidence of insecure handling)

  context_window_check

  input parameter "max_context_tokens" accepts a credential (informational; not evidence of insecure handling)

  build_rag_prompt

  input parameter "secret" accepts a credential (informational; not evidence of insecure handling)

  generate_hmac

  input parameter "api_key" accepts a credential (informational; not evidence of insecure handling)

  web_security_auditrun_vlm_test_suitemultimodal_eval_guiderun_semantic_teststest_skilljira_to_test_suitefix_gherkin

  input parameter "tokens" accepts a credential (informational; not evidence of insecure handling)

  needle_haystack_generate

  input parameter "tokens_per_day" accepts a credential (informational; not evidence of insecure handling)

  llm_fit_finder

  input parameter "api_keys" accepts a credential (informational; not evidence of insecure handling)

  run_eval_contractrun_vlm_test_suite_batch

  input parameter "jira_token" accepts a credential (informational; not evidence of insecure handling)

  post_jira_commentfetch_jira_issuesearch_jira_issuesjira_to_test_suite

  input parameter "confluence_token" accepts a credential (informational; not evidence of insecure handling)

  create_confluence_pagefetch_confluence_page

  Fix: Avoid accepting raw credentials as input parameters. Use secret managers (e.g. 1Password CLI, AWS Secrets Manager) and ensure credentials are never logged or stored in agent traces.

• InfoAS-006

  💻Arbitrary Code Execution ×3

  tool name or description implies arbitrary script/code execution — capability unconfirmed (no exec permission or code/script/eval input property found)

  mcp_server_evaluatererank_evaluateweb_security_audit

  Fix: This tool can execute arbitrary code or shell commands on the host system. Remove it unless strictly required. If kept: (1) restrict access to trusted users/agents only, (2) require human approval before each invocation (Claude Desktop: set approval_required: true; other clients: enable equivalent confirmation), (3) use the most restrictive sandbox or read-only mode available, and (4) never expose this tool to untrusted input sources.