Twitter Profile cap...
 
Notifications
Clear all

Twitter Profile capture

10 Posts
7 Users
0 Reactions
1,232 Views
ridders
(@ridders)
Active Member
Joined: 13 years ago
Posts: 12
Topic starter  

Hi,

Has anyone got any experience with exporting all the contents from a publically accessible twitter profile?

The problem I'm having is that if the webpage is saved off as an HTML file, then all the collapsed replies are not captured. I've attempted to use HTTrack, but the same issue occurs.

I also looked into using the twitter API, however it does not provide a timeline that includes replies from other users.

Does anyone know of a way of producing the timeline and include the replies in a nice readable format?


   
Quote
ridders
(@ridders)
Active Member
Joined: 13 years ago
Posts: 12
Topic starter  

Should anyone else have the same issue and need to capture a twitter timeline, I created a Ruby script which will produce a TSV file containing the users timeline, along with other useful information. No OAuth, or twitter account required to use this.

Source code is available at https://launchpad.net/warbler

Feel free to email me, if you need any assistance.


   
ReplyQuote
JonN
 JonN
(@jonn)
Trusted Member
Joined: 20 years ago
Posts: 73
 

Haven't tried it yet, but it sounds very useful.

Thanks


   
ReplyQuote
ridders
(@ridders)
Active Member
Joined: 13 years ago
Posts: 12
Topic starter  

If needed, I can convert it to EXE, just let me know.


   
ReplyQuote
(@jonathan)
Prominent Member
Joined: 20 years ago
Posts: 878
 

If needed, I can convert it to EXE, just let me know.

That might be handy - thanks.


   
ReplyQuote
(@alexc)
Reputable Member
Joined: 16 years ago
Posts: 301
 

I should learn some Ruby at some point, but coming from Python all of those "end"s make me sad.

Useful script, thanks for sharing )


   
ReplyQuote
EricZimmerman
(@ericzimmerman)
Estimable Member
Joined: 13 years ago
Posts: 222
 

related to this, if anyone ever does a search warrant for twitter (which is SO much fun to do because they make the process easy), i wrote a parser that will take their return data and turn it into something usable.

if anyone needs it lemme know


   
ReplyQuote
flamerescue150
(@flamerescue150)
Eminent Member
Joined: 19 years ago
Posts: 23
 

If you're willing to pay for it, X1 Social Discovery does a very nice job of capturing twitter content and allows for keyword searching, geo-mapping, and can create various report types. The nice part about this tool is that it will automatically generate .png images of all webpages linked to by the collected tweets and download videos for youtube links found in tweets.


   
ReplyQuote
ridders
(@ridders)
Active Member
Joined: 13 years ago
Posts: 12
Topic starter  

As asked for, the script has been converted to EXE and can be downloaded from https://sourceforge.net/projects/warbler-twitter/

suggestions for additional features are always welcome


   
ReplyQuote
Bulldawg
(@bulldawg)
Estimable Member
Joined: 13 years ago
Posts: 190
 

I'm demoing a product called X1 Social Discovery that does exactly that. They have a 14 day free trial.

http//www.x1discovery.com/social_discovery.html

Social media has been very helpful in fraud cases we've worked on, and I'm hoping this product will make that process more efficient.


   
ReplyQuote
Share: