Click here to Skip to main content
15,867,308 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Hey!

I am working on twitter analysis with python. I have the code running and everything looks good, however, when I add the tweet to the tweets file it repeats existing tweet, how can I fix that?



from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
import json
import sentmod as s




#consumer key, consumer secret, access token, access secret.
ckey= "PpwBFRmltifRyYA57YcYtD5Sg"
csecret="BGQGAGeGHQwjoEF5OW2LSfbvoGyDm6qEDNftQxsyZwNVgylj31"
atoken="953938307855069185-6gJncM43PJmIZAsuM8MVrvyzRBhBjus"
asecret="SfMuQo9UKrTY9jXiBjEE1S9NoreAE8HSx3tEfiSPqzTkU"

class listener(StreamListener):
    def on_data(self, data):
        all_data = json.loads(data)
        tweet = all_data["text"]
        sentiment_value, confidence = s.sentiment(tweet)
        tweets= open("tweets.txt","a",encoding="utf-8")
        tweets.write(tweet)
        tweets.write('\n\n\n')
        tweets.close()
        print(tweet,sentiment_value, confidence)
        if confidence*100 >= 60:
            output = open("twitter-out.txt","a")
            output.write(sentiment_value)
            output.write('\n\n\n')
            output.close()
            return True

    def on_error(self, status):
        print(status)

auth = OAuthHandler(ckey, csecret)
auth.set_access_token(atoken, asecret)

twitterStream = Stream(auth, listener())
twitterStream.filter(track=["Siemens"],languages=['en'])



Any help please?

What I have tried:

Normal loops but i don't know how to apply them here
Posted
Updated 30-Jan-18 22:42pm
v3
Comments
Richard MacCutchan 31-Jan-18 3:56am    
You need to provide more details, where do you think a loop is needed?
Member 13647869 31-Jan-18 4:44am    
This part:
def on_data(self, data):
all_data = json.loads(data)
tweet = all_data["text"]
sentiment_value, confidence = s.sentiment(tweet)
tweets= open("tweets.txt","a",encoding="utf-8")
tweets.write(tweet)
tweets.write('\n\n\n')
tweets.close()

is getting the tweets, saving them in a txt file and printing them, so the loop should be before that?
Richard MacCutchan 31-Jan-18 4:56am    
I think it all depends on what gets returned in the call to json.loads, and what information you get in the tweet object. You need to do some debugging to find out what data is being passed in to your function. I also looked at Streaming With Tweepy — tweepy 3.3.0 documentation[^], but it does not offer any clues.
Member 13647869 31-Jan-18 5:00am    
what is returned in json.loads is the live twitter stream of a keyword that I am searching for, i can see the outcome as tweets, and some of which are being repeated and i do not want that to happen
Richard MacCutchan 31-Jan-18 5:04am    
If you are receiving duplicate information then you need to look at the source of the data. We cannot guess what is being received by your program.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900