Smart Cart with Real-Time Object Detection and Billing System

The Smart Cart with Real-Time Object Detection and Billing System is an advanced automation solution developed for the retail industry to simplify and modernize the checkout process. This project brings together the power of computer vision, embedded systems, and graphical interfaces to create an innovative system capable of recognizing both packed and loose items in real-time. It effectively eliminates the need for manual item scanning or weighing, allowing customers to shop without the delays typically encountered at checkout counters.

At the heart of the system lies a Raspberry Pi that processes live video feeds captured by a webcam mounted on the cart. The system employs two YOLO (You Only Look Once) object detection models—one trained to detect packed items like snacks and beverages, and another trained for loose items such as fruits and vegetables. As the customer adds items to the cart, the Smart Cart system immediately identifies them, logs their names, and calculates their cost based on a preloaded price list.

For loose items that require weight measurement (e.g., apples, potatoes), a load cell connected to an Arduino microcontroller accurately measures the weight. This weight data is then sent via a serial connection to the Raspberry Pi for further processing. The system dynamically updates the total cost by referencing a price JSON file, ensuring that each item is correctly billed according to its quantity and price per unit.

This seamless integration between the hardware and software components allows the system to automate the billing process, which is displayed in real-time through a Tkinter-based graphical user interface (GUI). At the end of the shopping trip, the customer can check out by scanning a QR code generated by the system, which represents the total amount for all items. The Smart Cart is designed to make retail shopping faster, more accurate, and more convenient for both customers and store owners, significantly reducing queues at checkout and improving the overall customer experience.

Objectives

• Automate the Retail Checkout Process: The primary objective of this project is to automate the process of item detection, weighing, and billing, eliminating the need for human intervention.
• Real-Time Object Detection: The system leverages YOLO models to detect packed and loose items instantly as they are added to the cart.
• Accurate Weight Measurement: For loose items, the system uses a load cell connected to an Arduino to measure the weight and calculate the price accordingly.
• Simplify Payment Process: After the shopping is completed, a QR code representing the total bill is generated for fast, hassle-free payment.
• Improve Shopping Efficiency: By integrating real-time detection and automated billing, the Smart Cart significantly reduces checkout times, making the shopping experience more efficient for customers.

Key Features

1. Real-Time Object Detection with YOLO Models: The system uses two YOLO models—one for packed items and another for loose items—to analyze a live video feed from the cart's camera, identifying items instantly.
2. Weight Measurement for Loose Items: A load cell measures the weight of loose items (e.g., fruits, vegetables), and this data is transmitted to the Raspberry Pi for price calculation.
3. Automated Billing System: As items are detected and weighed, the system automatically calculates the total price and updates it in real-time on the GUI. The price list is stored in a JSON file, which is accessed to match item names with prices.
4. QR Code Generation for Payment: Once all items have been processed, the system generates a QR code that encodes the total bill, allowing the customer to scan and pay using any digital wallet.
5. Multithreading for Enhanced Performance: To ensure that the system remains responsive during real-time item detection and GUI updates, multithreading is employed. One thread handles the YOLO object detection, while another manages the GUI and billing updates.
6. Graphical User Interface (Tkinter): The user-friendly GUI provides a clear, real-time display of the items detected, their quantities, and the total bill. It also handles the checkout process and generates the QR code.

Application Areas

• Supermarkets and Grocery Stores: This system is ideal for automating the checkout process in supermarkets, particularly for self-checkout stations.
• Self-Checkout Kiosks: Can be integrated into self-checkout kiosks, where customers can scan and pay for items independently without the need for store staff intervention.
• Hypermarkets: Large retailers can use the Smart Cart system to streamline the checkout process during busy shopping periods, reducing queues and improving customer service.
• Farmers' Markets: The system can also be deployed at farmers' markets for weighing and billing fresh produce quickly and accurately.
• Retail Stores and Convenience Shops: Smaller stores or convenience shops can benefit from the system’s ability to automate the billing process, making transactions faster and more efficient.

Detailed Working of Smart Cart with Real-Time Object Detection and Billing System

The Smart Cart system is designed to function seamlessly in real-world retail environments by combining several technologies.

• YOLO Object Detection: As items are placed in the cart, a camera continuously captures live video feeds. These frames are processed by two YOLO models—one specialized for detecting packed items (like snacks, canned goods, etc.) and the other for identifying loose items (like fruits and vegetables). Once an item is detected, its name is matched against a price list stored in a JSON file.

• Weight Measurement: For loose items, which are typically priced by weight, the system uses a load cell connected to an Arduino. When loose items are placed in the cart, the load cell measures their weight, and the Arduino sends this data to the Raspberry Pi through a serial connection. The system then calculates the total cost based on the item’s weight and the price per unit.

• Tkinter GUI: The Raspberry Pi runs a Tkinter-based graphical interface that displays the live camera feed, the items being added to the cart, and a real-time breakdown of the total bill. The GUI is updated in real-time to reflect changes as items are detected or weighed.

• Automated Billing: Every time an item is added to the cart, the system references a JSON file that contains the pricing details for each item. The name of the detected item is matched against the JSON data, and the correct price is applied, whether based on weight (for loose items) or quantity (for packed items).

• QR Code Generation: Once the customer is ready to check out, the system calculates the total cost of all the items. A QR code is then generated using this total amount. The customer can simply scan the QR code with a mobile payment app to complete the transaction.

Modules Used to Make Smart Cart with Real-Time Object Detection and Billing System

1. YOLO Object Detection Models: The system uses two separate YOLO models—one for identifying packed items and another for detecting loose items.
2. Arduino and Load Cell for Weight Measurement: The load cell measures the weight of loose items, and the Arduino transmits this data to the Raspberry Pi. This module ensures that items priced by weight are accurately billed.
3. Tkinter GUI for User Interaction: A graphical interface built using Tkinter provides real-time updates on detected items, quantities, prices, and total costs. The GUI also facilitates checkout by generating the QR code.
4. QR Code Generator: This module converts the total bill into a QR code for easy payment, allowing the customer to pay with a mobile wallet app.
5. Multithreading for System Efficiency: The system employs multithreading to handle different tasks simultaneously—ensuring that the GUI remains responsive while the object detection and billing processes run in parallel.

Other Possible Projects Using the Smart Cart with Real-Time Object Detection and Billing System Project Kit

• Automated Inventory Tracking System: This system could be adapted for warehouses, where it could detect items and log them into an inventory database in real-time.
• Smart Vending Machine: A vending machine that uses object detection to recognize items selected by the customer and then automatically processes payment via a QR code.
• Garbage Sorting System: This project could be repurposed for waste management, where different types of waste are detected and sorted automatically.
• Automated Kitchen Inventory System: A version of the Smart Cart could be used in commercial kitchens to track food items, update inventory, and generate shopping lists.