Scrapy User’s Guide

RHET 7350: Independent Study
Automating Data Collection in Digital Rhetorics Research
Dr. Brian Ray

smartmockups_k156gwlx.jpg

I have always been interested in different forms of technology and how they can help us improve certain areas of our lives. After taking Dr. Ray's Research Methods course, I became interested in data scraping tools and how we could use them to improve research as it relates to the rhetoric field. This resulted in an independent study course with Dr. Ray. Part of the course included analyzing existing tools to see not only how easy they were to use, but how successful they were in terms of the data collected.

For my final project in the course, I created online documentation for the open source tool Scrapy. It's worth noting that although documentation on Scrapy already exists, it is geared toward users that are familiar with scraping tools and programming. My goal was to create documentation for students in a rhetoric or research course, who might not be familiar with online tools or the command line.

In order to create successful documentation for my imagined audience, I talked to a few classmates and had them walk through the different stages of my guide. I took their feedback into account, adding notes for clarity throughout the document and changing sections if needed.

Although I'm not a huge fan of screenshots in documentation, if I had to revise parts of this project, I would add some screenshots for additional context. Users tend to get overwhelmed when a page is too text-heavy, so adding images might help reassure the reader and guide them through the setup process.