Dear visitors and members, with the staff having moved on to other life interests and as a result of changes in people's internet usage habits, the CodeWalrus community have migrated almost entirely to Discord, IRC and WalrusIRC and is now essentially a place where to hang out, chat about gaming, programming, music, anime, and chill. As a result, even though the CW forums will remain open for posting, for faster TI, Casio or HP calculator help and news we recommend using the bilingual TI-Planet forums instead.

We encourage everyone to join our Discord server now.

* WalrusIRC

You need to have 5 posts and not be part of restricted usergroups in order to use the WalrusIRC embedded shoutbox. However, you can also access our IRC channel called #CodeWalrus via EFnet.

Author Topic: Rule34 image downloader  (Read 4043 times)

0 Members and 1 Guest are viewing this topic.

Offline gameblabla

  • Super User
  • Join Date: May 2015
  • Location:
  • Posts: 749
  • Post Rating Ratio: +15/-7
  • TI-nspire porter
Rule34 image downloader
« on: January 21, 2018, 01:03:38 am »
So i met a guy on Discord and he made this image file scrapper in Python :
https://github.com/sunx2/r34py

Unfortunately, reading the source code was like trying to de-obfuscate it.
Plus, i had to use proxychains to make it work over Tor.

Hence why i decided as an exercise to re-implement it in C.
The main issues i had were to implement a function to crawl over the html files and find the image links.
This was the first time i did something like this so i took me a few more hours.
I eventually decided i should look for a pattern that would allow me to find them links fairly easily.
Code: [Select]
<a href="http:///....>

I eventually shortened the code so it only looks at  "<", ":" and if it is preceded by "r" two characters before the link.
Believe or not it works and i can now download all of my pr0n goodness from the command line.

Then, i extended it so it downloads it from all the pages (just a loop really) and here are the results :
https://github.com/gameblabla/r34downloader_curl

It uses curl for downloading things. Once i work on the SDL gui interface, it could be ported to other platforms too.

Lemme now what you think about it. You must compile it from source.


  • Calculators owned: TI Nspire CX, TI-89

Online xlibman

  • Omni founder & CW co-founder
  • CodeWalrus Staff
  • Super User
  • Forum Maintenance
  • Original 5
  • CodeWalrus Supporter
  • *
  • Join Date: Nov 2014
  • Location: Quebec, Canada
  • Posts: 18969
  • Post Rating Ratio: +100/-5
    • dj_omnimaga
    • DJOmnimaga.music
    • @DJOmnimaga
    • dj_omnimaga
    • @DJOmnimaga
    • /u/DJ_Omnimaga
    • DJOmnimaga
    • 112/11286
    • @djomnimaga
    • @DJOmnimaga
    • DJ Omnimaga music store
  • Gender: Male
Re: Rule34 image downloader
« Reply #1 on: January 23, 2018, 09:12:54 pm »
Is the only content allowed rule 34/pr0n material? :P
  • Calculators owned: TI-57, 73, TI-80 (broken), TI-81, TI-82, TI-83, TI-83+ (broken), TI-83+ (broken), TI-83+SE (broken), TI-84+, TI-84+CSE, TI-84+CE, TI-85, TI-86, TI-89T, TI-92, TI-Nspire, TI-Nspire CX (semi-broken), HP 39gII, HP Prime, Casio fx-7000G, fx-7400G+, fx-7700GE, fx-9750G+, fx-9750GII, fx-9860G, cfx-9850G, FX-1.0+, fx-CG10, fx-CP400
  • Consoles, mobile devices and vintage computers owned: Samsung i5510, Nexus 5, Atari 2600, Lynx, SMS, Game Gear, Genesis, Dreamcast, NES, SNES, N64, GCN, Wii, Wii U, GBA, DS, 3DS, PS2, PS3, PS4, PSP, PSVita, XBox 360, XBOne

Bandcamp|Reverbnation|Facebook|Youtube|Twitter
Retired Omnimaga admin (2001-11) and editor (2012-14)

Offline _iPhoenix_

  • Super User
  • Join Date: Mar 2017
  • Location:
  • Posts: 575
  • Post Rating Ratio: +12/-1
  • █████ ▼ ♪ Best music ♫
    • @dj_iPhoenix
    • @UCytgMNPxAMDsxjimvxHf01w
    • @the-legend-of-iphoenix
  • Gender: Male
Re: Rule34 image downloader
« Reply #2 on: January 24, 2018, 12:08:05 am »
It's really easy to do it manually, too. Most OS's provide an option to download a webpage and it's assets, and you can filter by image type.
  • Calculators owned: A sentient TI-84+ CE, TI-85
  • Consoles, mobile devices and vintage computers owned: TI-99/4a home computer
Please spam here: https://legend-of-iphoenix.github.io/spam/

"walruses are better than tuxedo chickens, all hail the great :walrii:" ~ me

Offline gameblabla

  • Super User
  • Join Date: May 2015
  • Location:
  • Posts: 749
  • Post Rating Ratio: +15/-7
  • TI-nspire porter
Re: Rule34 image downloader
« Reply #3 on: January 24, 2018, 10:26:06 pm »
Is the only content allowed rule 34/pr0n material? :P
Well the website itself only allows that so yeah. But this could be adapted to other websites but most don't post direct links
to the full pictures so yeah...

It's really easy to do it manually, too. Most OS's provide an option to download a webpage and it's assets, and you can filter by image type.
Well yes, that is true but the point was to only download what i wanted, namely just the images themselves and not the thumbnails, ads or other random s***. I doubt the OSes allow that very easily.
  • Calculators owned: TI Nspire CX, TI-89

Offline gameblabla

  • Super User
  • Join Date: May 2015
  • Location:
  • Posts: 749
  • Post Rating Ratio: +15/-7
  • TI-nspire porter
Re: Rule34 image downloader
« Reply #4 on: January 25, 2018, 08:26:00 pm »
Well, i tried it with some other tags like Mario and i realized it would not pick up some images properly.
So i fixed that and it works properly now.
  • Calculators owned: TI Nspire CX, TI-89

 


You can also use the following HTML or bulletin board code to share it on your page or forum signature!


Also do not forget to check our affiliates below.
Planet Casio TI-Planet Calc.news BroniesQC BosaikNet Velocity Games