Skip to content

Latest commit

 

History

History
130 lines (80 loc) · 3.39 KB

README.md

File metadata and controls

130 lines (80 loc) · 3.39 KB

Sogrape Webscraper

Sogrape × 42 Porto: Hackathon

Web scraping website developed in a 3-day period, during the Hackathon promoted by Sogrape and 42 Porto (23 - 26 Oct. 2023).

Table of contents

Challenge · Prerequisites · Installation · Usage · Credits · License


💥 Challenge

Implementing a web scraping system able to collect wine prices from national online stores on a daily basis, and to develop an interactive and intuitive interface to display the same data.


Approach

Automated web scraping website that would allow Sogrape’s employees to easily compare pricing between several retailers for the same product, and to keep track of price fluctuations over time, in an automated manner.

Login screen Webscraper view

🧭 Usage

Prerequisites

Before you begin, ensure you have met the following requirements:


Installation

  1. Clone the repository
git clone git@github.com:teresa-chow/42-hackathon-sogrape.git

  1. Start Apache and MySQL in XAMPP

    • Launch XAMPP Control Panel
    • Click Start next to Apache and MySQL

  2. Configure your project

    • Copy your project files to the XAMPP web server directory (usually C:\xampp\htdocs\hack_dashboard on Windows)
    • Edit your project configuration files if necessary

  3. Install Python and required packages

    • Download and install Python;

    • Open the Windows command prompt

    • Navigate to your project directory

    • Install the required packages using pip

      python -m pip install -r requirements.txt

      or install them individually using

      pip install <package>

Usage

  1. Start XAMPP

    • Launch XAMPP Control Panel
    • Click Start next to Apache

  2. Run an SQL server instance and use our mydb_wine.sql file – this will ensure you are connected to a database that our program is compatible with; it can also run on any web server/host

  3. Access your PHP website

    • Ensure XAMPP is running
    • Click Admin button to acces the root/index of our website

  4. Run the Python code

    • Open a terminal/command prompt

    • Navigate to your project directory

    • Execute your Python script using the following command

      python main.py

  5. Refresh the dashboard using the button – all the data of the SQL will appear


Credits

Bernardo Esteves @berestv

Bruno Lopes @brpereiraa

João Ramalhosa @joaoped2-42PORTO

Ricardo Santos @rssantos342

Teresa Chow (me)

Vinicius Vaccari @vivaccar


License

This work is published under the terms of the MIT License.


⬆ back to top