Django Scraper Matching Issue: match_maker Only Returns 4 Members Instead of 150

Question:

I'm working on a Django project that uses Scrapy to scrape member profiles from a website. The scraped data is processed by a method called match_maker. However, I'm encountering an issue where match_maker only returns 4 members, despite having 150 members in the database (excluding 3 staff members).

Details:

Database: Contains 153 members; 3 are staff members, leaving 150 regular members. Profile Types: Each member has a profile_type of either 'Man', 'Woman', 'Trans', or 'Couple'.

Issue:

In the match_maker method, there's a loop that processes rooms and assigns them to members. A set named used_rooms is used to track assigned rooms to ensure each room is only assigned once. The relevant code snippet is:

if room["username"] in used_rooms:
    continue

When this condition is active, only 4 members are returned. If I comment out this check, the method processes all 150 members, but the number of available rooms exceeds one million, which is incorrect.

Objective:

I need each room to be assigned to only one member, ensuring no more than one member owns a particular room. I'm looking for guidance on how to resolve this issue so that match_maker correctly processes all 150 members without assigning multiple members to the same room.

What I've Tried:

Ensured Uniqueness: Verified that room["username"] is unique for each room. Debugged used_rooms: Printed the contents of used_rooms before and after the check to ensure it's being populated correctly. Checked Room Data Structure: Confirmed that room["username"] is unique across all rooms. Despite these efforts, the issue persists. Any insights or suggestions would be greatly appreciated.

Code snippet:

def match_maker(self, members, room_data: list):
        matched_items = []
        used_rooms = set() # Keep track of assigned rooms
        room_data_copy = deepcopy(room_data)
        
        # Group rooms by profile_type for quick lookup
        rooms_by_type = defaultdict(list)
        
        for room in room_data_copy:
            if len(room["body"]) >= 5:  # Only store rooms with a valid description
                rooms_by_type[room["profile_type"]].append(room)

        users = set()

        for member in members:
            if member.is_staff:
                continue
            
            # Get matching rooms for this profile_type
            available_rooms = rooms_by_type.get(member.profile_type, [])
                        
            for room in available_rooms:
                if room["id"] in used_rooms:
                    continue

                if room["profile_type"] != member.profile_type:
                    continue
                
                users.add(member.username)      
                matched_items.append((member, room))
                used_rooms.add(room["id"])
        
        print(f"The users are: {users}")
        print(f"The number of users are: {len(users)}")
        
        
        random.shuffle(matched_items)
        
        return matched_items

Вернуться на верх