more refactoring
This commit is contained in:
William Kray
+183
@@ -0,0 +1,183 @@
|
||||
# Community Bot Refactoring
|
||||
|
||||
This document describes the refactoring performed on the community bot project to improve code organization, maintainability, and testability.
|
||||
|
||||
## Overview
|
||||
|
||||
The original `bot.py` file contained over 3,800 lines of code with mixed concerns, making it difficult to maintain and test. The refactoring separates the code into logical modules and adds comprehensive test coverage.
|
||||
|
||||
## New Structure
|
||||
|
||||
### Helper Modules (`community/helpers/`)
|
||||
|
||||
The helper functions have been extracted into separate modules based on their functionality:
|
||||
|
||||
#### `message_utils.py`
|
||||
- `flag_message()` - Check if a message should be flagged for censorship
|
||||
- `flag_instaban()` - Check if a message should trigger instant ban
|
||||
- `censor_room()` - Check if a message should be censored based on room config
|
||||
- `sanitize_room_name()` - Sanitize room names for aliases
|
||||
- `generate_community_slug()` - Generate community slugs from names
|
||||
|
||||
#### `room_utils.py`
|
||||
- `validate_room_alias()` - Check if a room alias exists
|
||||
- `validate_room_aliases()` - Validate multiple room aliases
|
||||
- `get_room_version_and_creators()` - Get room version and creators
|
||||
- `is_modern_room_version()` - Check if room version is modern (12+)
|
||||
- `user_has_unlimited_power()` - Check if user has unlimited power
|
||||
- `get_moderators_and_above()` - Get users with moderator+ permissions
|
||||
|
||||
#### `user_utils.py`
|
||||
- `check_if_banned()` - Check if user is banned according to banlists
|
||||
- `get_banlist_roomids()` - Get room IDs for banlists
|
||||
- `ban_user_from_rooms()` - Ban user from multiple rooms
|
||||
- `user_permitted()` - Check if user has sufficient power level
|
||||
|
||||
#### `database_utils.py`
|
||||
- `get_messages_to_redact()` - Get messages to redact for a user
|
||||
- `redact_messages()` - Redact queued messages in a room
|
||||
- `upsert_user_timestamp()` - Insert/update user activity timestamp
|
||||
- `get_inactive_users()` - Get lists of inactive users
|
||||
- `cleanup_stale_verification_states()` - Clean up old verification states
|
||||
- `get_verification_state()` - Get verification state for DM room
|
||||
- `create_verification_state()` - Create new verification state
|
||||
- `update_verification_attempts()` - Update verification attempts
|
||||
- `delete_verification_state()` - Delete verification state
|
||||
|
||||
#### `report_utils.py`
|
||||
- `generate_activity_report()` - Generate activity report from DB results
|
||||
- `split_doctor_report()` - Split large reports into chunks
|
||||
- `format_ban_results()` - Format ban operation results
|
||||
- `format_sync_results()` - Format sync operation results
|
||||
|
||||
### Test Structure (`tests/`)
|
||||
|
||||
Comprehensive test coverage has been added for all modules:
|
||||
|
||||
#### `test_message_utils.py`
|
||||
- Tests for message flagging and censoring functions
|
||||
- Tests for room name sanitization and slug generation
|
||||
- Edge cases and error handling
|
||||
|
||||
#### `test_room_utils.py`
|
||||
- Tests for room alias validation
|
||||
- Tests for room version and creator detection
|
||||
- Tests for power level and permission checks
|
||||
|
||||
#### `test_user_utils.py`
|
||||
- Tests for ban checking and user banning
|
||||
- Tests for permission validation
|
||||
- Tests for banlist management
|
||||
|
||||
#### `test_database_utils.py`
|
||||
- Tests for database operations
|
||||
- Tests for message redaction
|
||||
- Tests for user activity tracking
|
||||
- Tests for verification state management
|
||||
|
||||
#### `test_report_utils.py`
|
||||
- Tests for report generation and formatting
|
||||
- Tests for report splitting and chunking
|
||||
- Tests for result formatting
|
||||
|
||||
#### `test_bot_commands.py`
|
||||
- Tests for all command handlers
|
||||
- Tests for permission checking
|
||||
- Tests for error handling
|
||||
|
||||
#### `test_bot_events.py`
|
||||
- Tests for all event handlers
|
||||
- Tests for proactive banning
|
||||
- Tests for power level synchronization
|
||||
- Tests for user activity tracking
|
||||
|
||||
## Benefits of Refactoring
|
||||
|
||||
### 1. **Improved Maintainability**
|
||||
- Code is now organized into logical modules
|
||||
- Each module has a single responsibility
|
||||
- Functions are smaller and more focused
|
||||
- Easier to locate and modify specific functionality
|
||||
|
||||
### 2. **Better Testability**
|
||||
- Each helper function can be tested independently
|
||||
- Mock objects can be easily injected for testing
|
||||
- Test coverage is comprehensive across all modules
|
||||
- Tests are organized by functionality
|
||||
|
||||
### 3. **Enhanced Readability**
|
||||
- Main bot class is now much smaller and focused
|
||||
- Helper functions have clear names and purposes
|
||||
- Code is easier to understand and follow
|
||||
- Documentation is improved with docstrings
|
||||
|
||||
### 4. **Reduced Complexity**
|
||||
- Complex functions have been broken down
|
||||
- Dependencies are clearer and more explicit
|
||||
- Code duplication has been eliminated
|
||||
- Error handling is more consistent
|
||||
|
||||
### 5. **Easier Debugging**
|
||||
- Issues can be isolated to specific modules
|
||||
- Functions are smaller and easier to debug
|
||||
- Test failures provide clear indication of problems
|
||||
- Logging is more targeted and useful
|
||||
|
||||
## Running Tests
|
||||
|
||||
### Prerequisites
|
||||
```bash
|
||||
pip install pytest
|
||||
```
|
||||
|
||||
### Run All Tests
|
||||
```bash
|
||||
python run_tests.py
|
||||
```
|
||||
|
||||
### Run Specific Test Module
|
||||
```bash
|
||||
pytest tests/test_message_utils.py -v
|
||||
```
|
||||
|
||||
### Run Tests with Coverage
|
||||
```bash
|
||||
pytest tests/ --cov=community --cov-report=html
|
||||
```
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### For Developers
|
||||
|
||||
1. **Import Changes**: Helper functions are now imported from their respective modules:
|
||||
```python
|
||||
from community.helpers import message_utils, room_utils, user_utils
|
||||
```
|
||||
|
||||
2. **Function Calls**: Helper functions now take explicit parameters instead of using `self`:
|
||||
```python
|
||||
# Old
|
||||
result = self.flag_message(msg)
|
||||
|
||||
# New
|
||||
result = message_utils.flag_message(msg, self.config["censor_wordlist"], self.config["censor_files"])
|
||||
```
|
||||
|
||||
3. **Testing**: New tests should be added to the appropriate test module in the `tests/` directory.
|
||||
|
||||
### For Users
|
||||
|
||||
The refactoring is completely transparent to end users. All commands and functionality remain exactly the same.
|
||||
|
||||
## Future Improvements
|
||||
|
||||
1. **Type Hints**: Add comprehensive type hints throughout the codebase
|
||||
2. **Async Context Managers**: Use async context managers for database operations
|
||||
3. **Configuration Validation**: Add configuration validation and schema
|
||||
4. **Logging Improvements**: Implement structured logging
|
||||
5. **Performance Monitoring**: Add performance metrics and monitoring
|
||||
6. **Documentation**: Generate API documentation from docstrings
|
||||
|
||||
## Conclusion
|
||||
|
||||
The refactoring significantly improves the codebase's maintainability, testability, and readability while preserving all existing functionality. The modular structure makes it easier to add new features, fix bugs, and ensure code quality through comprehensive testing.
|
||||
Reference in New Issue
Block a user