Click here to Skip to main content
15,889,266 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
First, I have try to scape data and I succefully done it. Next is store the information to a sqlite databse. I am stuck and not idea . I have using jsoup to web scrape data. I have try to using jdbc sqlite to create table. But I do not know how to connect each them like scaped data store in database. I have stuck in no 3, anyone please help me and guide me.Thanks advanced

The question is below:

You are required to develop a small system using MAVEN and Java programming language. The system should be able to:

1.Read Github-Id from a Github Issue page. Refer to the link below for the testing purpose.

https://github.com/STIW3054-A202/Main-Data/issues/1

2.Read a list of student from a Github Wiki page. Refer to the link below for the testing purpose.

https://github.com/STIW3054-A202/Main-Data/wiki/List_of_Student

3.Store the data into Sqlite database using the table format below:

| Matric | Name                         | GitHub Link               | Status |
|--------|------------------------------|---------------------------|--------|
| 243340 | Yu Zhixiong                  | https://github.com/abcde  | Yes    |
| 250634 | Ahmad Afham Bin Noor Azizan  |                           | No     |

4.Compare the data and produce the following reports:

List all students who have submitted the GitHub account.
List all students who have not submitted the GitHub account.
Reporting format:

List all students who have submitted the GitHub account:

| No. | Matric | Name                            | GitHub Link               |
|-----|--------|---------------------------------|---------------------------|
| 1   | 243340 | Yu Zhixiong                     | https://github.com/abcde  |
| 2   | 253242 | Umi Madihah Binti Mohamed Raimi | https://github.com/fghij  |


List all students who have NOT submitted the GitHub account:

| No. | Matric | Name                            |
|-----|--------|---------------------------------|
| 1   | 250634 | Ahmad Afham Bin Noor Azizan     | 
| 2   | 261307 | Lim Wei Yi                      | 


What I have tried:

package com;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;

public class escape {
    public static void main (String[] args)  {

        new escape().list();

    }
    public void list() {
        final String url =
                "https://github.com/STIW3054-A202/Main-Data/wiki/List_of_Student";

        try {
            final Document document = Jsoup.connect(url).get();

            for (Element row : document.select(
                    "div.gollum-markdown-content tr")) {
                if (row.select("td:nth-of-type(1)").text().equals("")) {
                    continue;
                } else {
                    final String ticker =
                            row.select("td:nth-of-type(1)").text();
                    final String matric_no =
                            row.select("td:nth-of-type(2)").text();
                    final String name =
                            row.select("td:nth-of-type(3)").text();

                    System.out.println(ticker + " " + matric_no + " " + name);
                }
            }
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }

}



package com;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;

/**
 *
 * @author sqlitetutorial.net
 */
public class insert {

    /**
     * Connect to the test.db database
     *
     * @return the Connection object
     */
    private Connection connect() {
        // SQLite connection string
        String url = "jdbc:sqlite:C://sqlite/db/test.db";
        Connection conn = null;
        try {
            conn = DriverManager.getConnection(url);
        } catch (SQLException e) {
            System.out.println(e.getMessage());
        }
        return conn;
    }

    /**
     * Insert a new row into the warehouses table
     *
     //* @param name
    // * @param capacity
     */
    public void insertdata (String name, double capacity) {
        String sql = "INSERT INTO warehouses(name,capacity) VALUES(?,?)";

        try (Connection conn = this.connect();
             PreparedStatement pstmt = conn.prepareStatement(sql)) {
            pstmt.setString(1, name);
            pstmt.setDouble(2, capacity);
            pstmt.executeUpdate();
        } catch (SQLException e) {
            System.out.println(e.getMessage());
        }
    }

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {

        insert app = new insert();
        // insert three new rows
        app.insertdata("Raw Materials", 3000);
        app.insertdata("Semifinished Goods", 4000);
        app.insertdata("Finished Goods", 5000);

    }

}


1.This code that I scrape data, and succefull run out a list student .Now i want store this in a sqlite table. I donnt know how to connect scrapped data to store in sqlite.

2.I have try do sqlite databse .But i can do in normal way and not idea how to the use scaped data to store.

3. According question there i have link is about githublink .we need this github link and join the student list to make a table ,like in the no 3 question output.
Posted
Updated 5-Apr-21 3:58am
v5

1 solution

 
Share this answer
 
Comments
Cow cow Lee 5-Apr-21 4:40am    
Hi thanks for answer ,but this website i have read before.So according this website i try to using jdbc sqlite .And i can insert normally like what input i type.
But now i have problem is scaped data store in this sqlite table.
Richard MacCutchan 5-Apr-21 5:09am    
Sorry, but we cannot guess what you are doing or what problems you are having. Please use the Improve question link above, show your code and explain what is wrong.
Cow cow Lee 5-Apr-21 7:29am    
I already update, it is clear?
Richard MacCutchan 5-Apr-21 7:34am    
The information that you are printing you can also store in your database.
Cow cow Lee 5-Apr-21 7:40am    
so sorry it is give me a link or document because i not see anything ? or you ask me a question.
I update the code that i using in scrape data, but it cannot store in a dabse

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900