delet­ing twit­ter like a nerd

Intro

for a long time I’ve want­ed to move on to the indieweb/​fediverse world from twit­ter and co. I fig­ured delet­ing my tweets would be a great way­point for my tran­si­tion¹. Twitter did­n’t pro­vide any tools for batch dele­tion, and all the online twit­ter dele­tion ser­vices only allowed a lim­it­ed num­ber of dele­tions on their free tier². So I decid­ed to do some good old snoop­ing and try to do it myself. It turned out to be much eas­i­er than I’d antic­i­pat­ed³.

Commencing Investigation

Open your browser’s net­work log pan­el (open dev­tools and find the net­work tab), you can see all the ongo­ing requests that your cur­rent­ly opened web page is mak­ing. If you delete a tweet, you’ll see a POST request is made to a DeleteTweet graphql endpoint.

Let’s inves­ti­gate this request. (I will be hid­ing val­ues in sub­se­quent images and snip­pets, which I thought might be sensitive.)

It looks like Twitter is cre­at­ing an authen­ti­ca­tion token, prob­a­bly from your login cook­ies. That’s actu­al­ly help­ful because we don’t have to wor­ry about cre­at­ing a token our­selves, we already got a head­er served on a plate.

Also, check the pay­load that is being sent in the request, and you’ll see a twitter_​id being sent, which iden­ti­fies which tweet to delete.

Okay, very infor­ma­tive stuff so far, right? You can find an inter­est­ing menu item in your browser’s devel­op­er tools, that allows you to copy a request as a self-con­tained curl command:

You will get a curl com­mand like this, all head­ers already baked in for your happiness:

curl 'https://api.twitter.com/graphql/[REDACTED]/DeleteTweet'
  -X POST
  -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0'
  -H 'Accept: */*'
  -H 'Accept-Language: en-US,en;q=0.5'
  -H 'Accept-Encoding: gzip, deflate, br'
  -H 'Content-Type: application/json'
  -H 'Referer: https://twitter.com/'
  -H 'x-twitter-auth-type: OAuth2Session'
  -H 'x-twitter-client-language: en'
  -H 'x-twitter-active-user: yes'
  -H 'x-csrf-token: [REDACTED]'
  -H 'Origin: https://twitter.com'
  -H 'Sec-Fetch-Dest: empty'
  -H 'Sec-Fetch-Mode: cors'
  -H 'Sec-Fetch-Site: same-site'
  -H 'DNT: 1'
  -H 'Sec-GPC: 1'
  -H 'authorization: Bearer [REDACTED]'
  -H 'Connection: keep-alive'
  -H 'Cookie: [REDACTED]'
  -H 'Pragma: no-cache'
  -H 'Cache-Control: no-cache'
  -H 'TE: trailers'
  --data-raw '{"variables":{"tweet_id":"863987907815546881","dark_request":false},"queryId":"[REDACTED]"}'

The request pay­load is sent with the --data-raw parameter.

You already know where this is going right? we can just run this curl com­mand for every tweet we want to delete, and we are done! But how do you find the tweet_​ids?

Collecting tweet IDs

Twitter will con­ve­nient­ly give you a list of all your tweet IDs (among a whole bunch of oth­er things) if you request an archive of your twit­ter data.

Go to https://​twit​ter​.com/​s​e​t​t​i​n​g​s​/​d​o​w​n​l​o​a​d​_​y​o​u​r​_​d​ata⁴. You will get a pass­word ver­i­fi­ca­tion prompt and after/​if you pass this screen⁵, Twitter will start cre­at­ing an archive of your data, and you should get a noti­fi­ca­tion to down­load the archive with­in 24 hours, assum­ing Twitter is still work­ing some­what okay.

Download this archive, and extract the tweets.js file some­where⁶. If you open this file, you will see a bunch of attrib­ut­es for all your tweets as a gigan­tic list of javascript objects, among them will be an id field. For example:

window.YTD.tweets.part0 = [
  {
    "tweet" : {
      "edit_info" : {
        "initial" : {
          "editTweetIds" : [
            "774174446479290368"
          ],
          "editableUntil" : "2016-09-09T09:45:46.163Z",
          "editsRemaining" : "5",
          "isEditEligible" : true
        }
      },
      "retweeted" : false,
      "source" : "<a href=\"http://twitter.com/download/android\" rel=\"nofollow\">Twitter for Android</a>",
      "entities" : {
        "hashtags" : [ ],
        "symbols" : [ ],
        "user_mentions" : [ ],
        "urls" : [ ]
      },
      "display_text_range" : [
        "0",
        "138"
      ],
      "favorite_count" : "0",
      "id_str" : "774174446479290368",
      "truncated" : false,
      "retweet_count" : "0",
      "id" : "774174446479290368",
      "created_at" : "Fri Sep 09 09:15:46 +0000 2016",
      "favorited" : false,
      "full_text" : "A common joke evolved: there are 11 types of people, those who know binary, those who don't, and those who don't know what's 3 in binary 😄",
      "lang" : "en"
    }
  },
  /* more "tweet" object blocks */
]

There will be a series of these tweet objects, one for each of your tweets.

Now you might think we can eas­i­ly process these with jq, but look again at the first line: this file declares the data as a javascript object vari­able, which is not valid JSON and hence unread­able to jq.

cat tweets.js | jq -r '.[]'
# parse error: Invalid numeric literal at line 1, column 24

Let’s do some com­mand-line surgery on that first line⁷:

cat tweets.js | awk '{if (NR==1) print "["; else print $0}'

This pipeline replaces the vari­able dec­la­ra­tion in the first line with and array start token "[", which in turn makes this whole thing valid JSON.

[
  {
    "tweet" : {
      /* ... */
    }
  },
  /* ... */
]   

Anyway, now we can extract the IDs like so:

cat tweets.js | awk '{if (NR==1) print "["; else print $0}' | jq -r '.[] | .tweet.id'

which returns a nice list of IDs:

825335392257843201
825194054178664448
823385342837305345
...

All Together Now

Now lets com­bine all these parts togeth­er into a loop­ing shell script. Copy the curl com­mand for /DeleteTweet from your browser’s net­work pan­el, then run it in loop over the tweet IDs from jq:

for TWEET_ID in $(cat tweets.js | awk '{if (NR==1) print "["; else print $0}' | jq -r '.[] | .tweet.id'); do
  curl -s 'https://api.twitter.com/graphql/[REDACTED]/DeleteTweet' -X POST
    # ... hidden  for brevity
    -H 'Accept-Encoding: identity'
    # ... hidden  for brevity
    --data-raw '{"variables":{"tweet_id":"' $TWEET_ID '","dark_request":false},"queryId":"[REDACTED]"}';
  echo '';
done;

Note that I made some changes:

• insert­ed $TWEET_ID into the --data-raw parameter
• mod­i­fied the Accept-Encoding Header val­ue to identity so that respons­es are uncom­pressed plain text⁸
• added the -s switch to curl to hide request progress
• added an emp­ty echo '' com­mand to sep­a­rate each curl out­put by a new line.

After run­ning this loop, you should start see­ing out­puts like this:

{"data":{"delete_tweet":{"tweet_results":{}}}}
{"data":{"delete_tweet":{"tweet_results":{}}}}
{"data":{"delete_tweet":{"tweet_results":{}}}}
{"data":{"delete_tweet":{"tweet_results":{}}}}
...

Let it keep run­ning, and refresh your twit­ter pro­file page to see your tweet count going down to!

Some notes

All this might seem very com­pli­cat­ed, but writ­ing this blog post took much longer than this entire inves­ti­ga­tion and exe­cu­tion process, and def­i­nite­ly much faster than using Twitter’s APIs as intend­ed (are Twitter APIs paid yet?), with­out hav­ing to pay any­body any­thing. At least until Twitter reads my post (unlike­ly) and puts roadblocks.

Although there may be some edge cas­es. For exam­ple, the tweets.js file calls the tweet object window.YTD.tweets.part0 which hints that they might seg­ment tweets into sep­a­rate vari­ables or files if the tweet count cross­es some lim­its; authen­ti­ca­tion tokens might need to be refreshed peri­od­i­cal­ly, etc.

Curiously, after going through this process, my twit­ter time­line became emp­ty, but the head­line is still show­ing I have 8 tweets. What kind of lim­bo are these 8 tweets going through?

This arti­cle was moti­vat­ed by The Penguins Club BlogTalk event. #blogtalk #pen­guin­sclub