OAuth Password Grant Flow Implementation - closes #1109

[crivetimihai]

OAuth Password Grant Flow Implementation - closes #1109

Overview

Implemented complete support for OAuth 2.0 Resource Owner Password Credentials Grant (RFC 6749 Section 4.3) to enable Keycloak-style authentication integration.

Fixes: #1109

Problem Statement

Users could not integrate MCP Gateway with Keycloak OAuth2 endpoints because:

1. Backend Limitation: Only client_credentials and authorization_code grant types were supported
2. Missing Protocol Support: No support for application/x-www-form-urlencoded request format required by Keycloak
3. UI Gap: No UI option to select password grant type
4. Form Fields Missing: No username/password input fields in the UI
5. Backend Processing: Form handlers didn't process username/password credentials

This caused errors like:

400 Bad Request: {"error":"invalid_request","error_description":"Missing form parameter: grant_type"}

Solution Implemented

1. Backend Implementation

OAuth Manager (mcpgateway/services/oauth_manager.py)

Lines 185-186: Added password grant type support

if grant_type == "password":
    return await self._password_flow(credentials)

Lines 287-384: Implemented _password_flow() method

• Validates username and password are present
• Decrypts client secret if encrypted
• Sends application/x-www-form-urlencoded request via aiohttp data parameter
• Handles both JSON and form-urlencoded responses
• Includes retry logic with exponential backoff
• Proper error handling and logging

Key Features:

• Required fields: username, password, token_url
• Optional fields: client_id, client_secret, scopes
• Automatic form encoding via aiohttp
• Response format flexibility (JSON or form-encoded)

2. Unit Tests

Test File (tests/unit/mcpgateway/test_oauth_manager.py)

Lines 83-184: Added 3 comprehensive test cases

1. test_get_access_token_password_flow_success (lines 84-135)

   • Tests successful token acquisition with JSON response
   • Verifies form data is correctly assembled
   • Validates all parameters are included in request

2. test_get_access_token_password_flow_missing_credentials (lines 137-149)

   • Tests error handling when username/password missing
   • Verifies appropriate OAuthError is raised

3. test_get_access_token_password_flow_form_urlencoded_response (lines 151-184)

   • Tests handling of form-urlencoded responses
   • Validates response parsing for non-JSON formats

Test Results:

• All 119 OAuth manager tests passing
• Coverage: 69% for oauth_manager.py (up from 12%)

3. UI Components

HTML Template (mcpgateway/templates/admin.html)

Create Gateway Form:

• Lines 4567-4569: Added password grant option to dropdown

  <option value="password">
    Resource Owner Password Credentials (Keycloak/Legacy)
  </option>

• Lines 4624-4640: Username input field

  • ID: oauth-username-field-gw
  • Name: oauth_username
  • Hidden by default (display: none)
  • Required when password grant selected

• Lines 4642-4658: Password input field

  • ID: oauth-password-field-gw
  • Name: oauth_password
  • Hidden by default (display: none)
  • Required when password grant selected

Edit Gateway Form:

• Lines 7258-7260: Password grant option in dropdown
• Lines 7357-7373: Username field (edit form)
  • ID: oauth-username-field-edit
• Lines 7375-7391: Password field (edit form)
  • ID: oauth-password-field-edit

4. JavaScript Logic

Admin Script (mcpgateway/static/admin.js)

Create Form Handler (handleOAuthGrantTypeChange):

• Lines 9309-9310: Get username and password field elements
• Lines 9341-9353: Show fields and make required when password grant selected

  if (grantType === "password") {
      usernameField.style.display = "block";
      passwordField.style.display = "block";
      usernameInput.required = true;
      passwordInput.required = true;
  }

• Lines 9355-9362: Hide fields and remove required for other grant types

Edit Form Handler (handleEditOAuthGrantTypeChange):

• Lines 9370-9371: Get field elements for edit form
• Lines 9402-9414: Show/require fields for password grant
• Lines 9416-9423: Hide/unrequire fields for other grants

5. Backend Form Processing

Admin Routes (mcpgateway/admin.py)

Create Gateway Handler:

• Line 5777: Extract oauth_username from form
• Line 5778: Extract oauth_password from form
• Line 5804: Add username to oauth_config
• Line 5806: Add password to oauth_config

Edit Gateway Handler:

• Line 6088: Extract oauth_username from form (edit)
• Line 6089: Extract oauth_password from form (edit)
• Line 6115: Add username to oauth_config (edit)
• Line 6117: Add password to oauth_config (edit)

Files Modified

File	Lines Changed	Purpose
mcpgateway/services/oauth_manager.py	185-186, 287-384	Password grant backend implementation
tests/unit/mcpgateway/test_oauth_manager.py	83-184	Unit tests for password flow
mcpgateway/templates/admin.html	4567-4658, 7258-7391	UI dropdowns and input fields
mcpgateway/static/admin.js	9306-9426	Field visibility and validation logic
mcpgateway/admin.py	5777-5806, 6088-6117	Form data processing

Total Lines Changed: ~250

Usage Example

Configuration via UI

1. Navigate to Gateway creation/edit form
2. Select "Resource Owner Password Credentials (Keycloak/Legacy)" from grant type dropdown
3. Username and password fields appear automatically
4. Fill in required fields:
   • Token URL: https://keycloak.example.com/auth/realms/myrealm/protocol/openid-connect/token
   • Username: systemadmin@system.com
   • Password: ********
   • Client ID (optional): mcp-gateway-client
   • Scopes (optional): openid profile email
5. Submit form

Configuration via API

{
  "name": "Keycloak Gateway",
  "url": "https://api.example.com",
  "oauth_config": {
    "grant_type": "password",
    "token_url": "https://keycloak.example.com/auth/realms/myrealm/protocol/openid-connect/token",
    "username": "systemadmin@system.com",
    "password": "my-secret-password",
    "client_id": "mcp-gateway-client",
    "scopes": ["openid", "profile", "email"]
  }
}

Token Request Format

The backend automatically sends:

POST /auth/realms/myrealm/protocol/openid-connect/token
Content-Type: application/x-www-form-urlencoded

grant_type=password&username=systemadmin@system.com&password=my-secret-password&client_id=mcp-gateway-client&scope=openid+profile+email

Testing

Unit Tests

# Run all OAuth manager tests
pytest tests/unit/mcpgateway/test_oauth_manager.py -v

# Run password flow tests specifically
pytest tests/unit/mcpgateway/test_oauth_manager.py::TestOAuthManager::test_get_access_token_password_flow_success -v
pytest tests/unit/mcpgateway/test_oauth_manager.py::TestOAuthManager::test_get_access_token_password_flow_missing_credentials -v
pytest tests/unit/mcpgateway/test_oauth_manager.py::TestOAuthManager::test_get_access_token_password_flow_form_urlencoded_response -v

Code Quality

# Format code
make autoflake isort black

# Run quality checks
make flake8 bandit pylint

# Security scan
bandit -r mcpgateway/services/oauth_manager.py

Results

• ✅ All 119 OAuth manager tests passing
• ✅ All 22 PKCE OAuth tests passing
• ✅ All 18 OAuth router tests passing
• ✅ Pylint: 10.00/10
• ✅ Flake8: Clean
• ✅ Bandit: No security issues
• ✅ Coverage: 69% for oauth_manager.py

Security Considerations

1. Password Handling: Passwords are only stored in oauth_config, not separately encrypted in the database (relying on database-level encryption)
2. Client Secret Encryption: Client secrets are encrypted using the auth_encryption_secret
3. No Password Logging: Passwords are never logged (only presence is logged)
4. Form Validation: Username and password are required fields when password grant is selected
5. HTTPS Enforcement: Token requests should only be made over HTTPS in production

Backwards Compatibility

• ✅ No breaking changes to existing OAuth flows
• ✅ client_credentials and authorization_code flows unchanged
• ✅ Existing gateways continue to work without modification
• ✅ New password grant type is opt-in via UI dropdown selection

Known Limitations

1. Password Grant Security: The password grant type is considered legacy and less secure than authorization_code flow. It should only be used for trusted first-party applications or when required by legacy systems like Keycloak.

2. Password Storage: Passwords are stored in the gateway's oauth_config. For production use, consider:

   • Using client_credentials or authorization_code flows when possible
   • Implementing additional encryption for oauth_config
   • Regular credential rotation policies

3. No Refresh Token UI: While the backend supports refresh tokens, there's no UI to manually trigger token refresh (it happens automatically when tokens expire)

Future Enhancements

• Add UI indicator for token expiration status
• Add manual token refresh button in UI
• Add support for additional OAuth grant types (implicit, device code)
• Enhanced password field encryption at rest
• OAuth token rotation policies UI

Verification Checklist

• Backend password flow implementation complete
• Unit tests added and passing
• UI dropdown has password option (create + edit)
• UI username/password fields present (create + edit)
• JavaScript field toggling works (create + edit)
• Backend form processing handles credentials (create + edit)
• All existing tests still pass
• Code quality checks pass (pylint, flake8, bandit)
• No syntax errors
• Element IDs match between HTML/JS
• Form field names match backend processing
• Integration flow validated end-to-end

References

• Issue: [Bug]:MCP Gateway UI OAuth2 Integration Fails with Keycloak Due to Missing x-www-form-urlencoded Support #1109
• OAuth 2.0 RFC: RFC 6749 Section 4.3 - Resource Owner Password Credentials Grant
• Keycloak Docs: Resource Owner Password Credentials
• aiohttp Docs: Form Data Encoding