Recently, I used python scrapy to crawl article's information like 'title' from a blog. I can have a good result without using a database. When I used sqlalchemy, I received an error displayed in the followed:
InterfaceError:(sqlite3.InterfaceError) Error binding parameter 0 -probably unsupported tpye.
[SQL: u'INSERT INTO myblog (title) VALUE (?)'] [parameters : ([u'\r\n Accelerated C++\u5b66\u4e60\u7b14 chapter 2 \r\n'],)]
and my xpath expression is:
from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
DeclarativeBase = declarative_base()
def db_connect():
return create_engine('sqlite:///./sqlalchemy.db?charset=utf-8',echo = True)
def create_myblog_table(engine):
DeclarativeBase.metadata.create_all(engine)
class MyBlog(DeclarativeBase):
__tablename__ = 'myblog'
id = Column(Integer, primary_key = True)
title = Column('title', String(200))
and pipeline's content followed:
from sqlalchemy.orm import sessionmaker
from model import MyBlog, db_connect, create_myblog_table
class MyblogPipeline(object):
def __init__(self):
engine = db_connect()
create_myblog_table(engine)
self.Session = sessionmaker(bind = engine)
def process_item(self, item, spider):
session = self.Session()
myblog=MyBlog(**item)
session.add(myblog)
session.commit()
return item
I know it remains me that the type of item['title'] is not supported by sqlite3.The content of item['title'] is [u'\r\n Accelerated c++\u5b66\u4e60 chapter3 ----- \u7528\u6279\u636e \r\n '] ,it's unicode,why sqlite3 doesn't support it?This blog's title information contains some Chinese,I wonder wether my definition of 'title' is string that caused this problem?Because my I am a tiro of sqlalchemy,I referred its documents, but found nothing, will you help me? I have no idea know!