* WalrusIRC

You need to have 5 posts and not be part of restricted usergroups in order to use the WalrusIRC embedded shoutbox. However, you can also access our IRC channel called #CodeWalrus via EFnet.

Author Topic: Pollwalrus API and Desktop Post Notifier  (Read 2005 times)

0 Members and 1 Guest are viewing this topic.

Offline DarkestEx

  • Super User
  • Join Date: Jan 2015
  • Location: Germany
  • Posts: 1318
  • Post Rating Ratio: +10/-2
    • @0xbmuessig
    • @muessigb
    • View Profile
    • My homepage
  • Gender: Male
Pollwalrus API and Desktop Post Notifier
« on: May 11, 2015, 09:24:04 pm »
Hello :walrii: sses!
I am working on a convenient API and desktop notifier for post replies.
The API is currently located here: http://api.muessigb.net/walrus_notify.php.
It outputs data using minified json and reads a number of get parameters over http.
If the client supports it, the data is sent using gzip compression.
The API is publicly usable for anybody who wants to make a post notifier for CW and should really not be overused (normal polling every minute is okay), but I would really like you to download it instead and upload it to your own server if you go below this refresh rate.
The API does alot of things: It downloads and parses the feed (saving you the trouble to configure the feed and saves you some traffic), makes the data more machine readable, converts the relative dates to unix timestamps, converts the relative links to absolute ones, parses the HTML and does one of three things (mode is selectable over get parameter):  1: Simply strip all html;  2: Strip all html, but preserve convert smileys;  3: Strip all html but convert smileys and convert tags like b and i to ** and // markup;  4: Leave the html intact
The api makes polling easier and you can ask the API if any updates happened since a specific post id. Also you can specify how many posts are checked or returned.

The desktop notifier will be using this API and refreshes between twice a minute (only for me), every minute or every 15 minutes.
It is going to get a similar design to the one of @Cumred_Snektron

More details will follow.


  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc
  • Consoles, mobile devices and vintage computers owned: Hewlett-Packard 100LX, Original Commodore 64C, Tektronix AWG2021, IBM X60s, IBM X60t, Nintendo DS Lite, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, Palm m5

Online DJ Omnimaga

  • Omni founder & CW co-founder
  • CodeWalrus Staff
  • Super User
  • Forum Maintenance
  • Original 5
  • CodeWalrus Supporter
  • *
  • Topic Management
  • Join Date: Nov 2014
  • Location: Quebec, Canada
  • Posts: 18179
  • Post Rating Ratio: +91/-4
    • dj_omnimaga
    • DJOmnimaga.music
    • @DJOmnimaga
    • dj_omnimaga
    • @DJOmnimaga
    • /u/DJ_Omnimaga
    • DJOmnimaga
    • 112/11286
    • @djomnimaga
    • @DJOmnimaga
    • View Profile
    • DJ Omnimaga music store
  • Gender: Male
Re: Pollwalrus API and Desktop Post Notifier
« Reply #1 on: May 11, 2015, 10:03:38 pm »
Does this fetch the site RSS feed every 1-15 minutes? Because everytime the RSS feed is open, it increases the board stats view count by 1, so if your tool opens it instead of using an SMF mod like Cumred's post notifier, then a refresh every minute will inflate our site view count by 1440 everyday. Not that this would be a serious issue, but it would kinda make our site traffic stats look less legit.

However, if by same design as Cumred's, you meant that it won't do that, then ignore my post. :P I think a Windows or Linux post notifier would be nice, in addition to the Android one. I will most likely add download links to them on http://codewalr.us/index.php?page=recent2 if they are hosted on our server and fully reliable, useability and resource-usage-wise.
  • Calculators owned: TI-73, TI-80 (broken), TI-81, TI-82, TI-83, TI-83+ (broken), TI-83+ (broken), TI-83+SE (broken), TI-84+, TI-84+CSE, TI-84+CE, TI-85, TI-86, TI-89T, TI-92, TI-Nspire, TI-Nspire CX (semi-broken), HP 39gII, HP Prime, Casio fx-7000G, fx-7400G+, fx-7700GE, fx-9750G+, fx-9750GII, fx-9860G, cfx-9850G, FX-1.0+, fx-CG10, fx-CP400
  • Consoles, mobile devices and vintage computers owned: Samsung i5510, Nexus 5, Atari 2600, Lynx, SMS, Game Gear, Genesis, Dreamcast, NES, SNES, N64, GCN, Wii, Wii U, GBA, DS, 3DS, PS2, PS3, PS4, PSP, PSVita, XBox 360, XBOne

Bandcamp|Reverbnation|Facebook|Youtube|Twitter
Retired Omnimaga admin (2001-11) and editor (2012-14)
??? ??? ??? ???

Offline Snektron

  • Lvl 69 Russian Snake
  • CodeWalrus Staff
  • Super User
  • Topic Management
  • Join Date: Dec 2014
  • Location: Netherlands
  • Posts: 3155
  • Post Rating Ratio: +31/-0
  • SSSssssss.....
    • RobinDeWalvis
    • Kzyrox
    • RobinDeWalvis
    • View Profile
    • quantuminfinity
  • Gender: Male
Re: Pollwalrus API and Desktop Post Notifier
« Reply #2 on: May 11, 2015, 10:17:18 pm »
Hmmm, we should combine the backends. Add a json subaction, while i keep my own low-data format. This way the notifiers will use only one server side resource. Very cool done though :)
  • Calculators owned: TI-84+
Legends say if you spam more than DJ Omnimaga, you will become a walrus...


Online DJ Omnimaga

  • Omni founder & CW co-founder
  • CodeWalrus Staff
  • Super User
  • Forum Maintenance
  • Original 5
  • CodeWalrus Supporter
  • *
  • Topic Management
  • Join Date: Nov 2014
  • Location: Quebec, Canada
  • Posts: 18179
  • Post Rating Ratio: +91/-4
    • dj_omnimaga
    • DJOmnimaga.music
    • @DJOmnimaga
    • dj_omnimaga
    • @DJOmnimaga
    • /u/DJ_Omnimaga
    • DJOmnimaga
    • 112/11286
    • @djomnimaga
    • @DJOmnimaga
    • View Profile
    • DJ Omnimaga music store
  • Gender: Male
Re: Pollwalrus API and Desktop Post Notifier
« Reply #3 on: May 11, 2015, 10:18:35 pm »
Actually that would be a good idea, so that we don't have to install two SMF mods for them (which is annoying due to mod conflicts).
  • Calculators owned: TI-73, TI-80 (broken), TI-81, TI-82, TI-83, TI-83+ (broken), TI-83+ (broken), TI-83+SE (broken), TI-84+, TI-84+CSE, TI-84+CE, TI-85, TI-86, TI-89T, TI-92, TI-Nspire, TI-Nspire CX (semi-broken), HP 39gII, HP Prime, Casio fx-7000G, fx-7400G+, fx-7700GE, fx-9750G+, fx-9750GII, fx-9860G, cfx-9850G, FX-1.0+, fx-CG10, fx-CP400
  • Consoles, mobile devices and vintage computers owned: Samsung i5510, Nexus 5, Atari 2600, Lynx, SMS, Game Gear, Genesis, Dreamcast, NES, SNES, N64, GCN, Wii, Wii U, GBA, DS, 3DS, PS2, PS3, PS4, PSP, PSVita, XBox 360, XBOne

Bandcamp|Reverbnation|Facebook|Youtube|Twitter
Retired Omnimaga admin (2001-11) and editor (2012-14)
??? ??? ??? ???

Offline DarkestEx

  • Super User
  • Join Date: Jan 2015
  • Location: Germany
  • Posts: 1318
  • Post Rating Ratio: +10/-2
    • @0xbmuessig
    • @muessigb
    • View Profile
    • My homepage
  • Gender: Male
Re: Pollwalrus API and Desktop Post Notifier
« Reply #4 on: May 11, 2015, 10:28:11 pm »
Does this fetch the site RSS feed every 1-15 minutes? Because everytime the RSS feed is open, it increases the board stats view count by 1, so if your tool opens it instead of using an SMF mod like Cumred's post notifier, then a refresh every minute will inflate our site view count by 1440 everyday. Not that this would be a serious issue, but it would kinda make our site traffic stats look less legit.

However, if by same design as Cumred's, you meant that it won't do that, then ignore my post. :P I think a Windows or Linux post notifier would be nice, in addition to the Android one. I will most likely add download links to them on http://codewalr.us/index.php?page=recent2 if they are hosted on our server and fully reliable, useability and resource-usage-wise.
Currently I am using the SMF RSS feed and alot of PHP ninja skillz to make it work. The feed is fetched on every API request and then processed on my server (produces quite some load due to gzip compression). The polling can be done in any possible rate. Also I was planning to cache the response unless thats overridden using a get parameter.
Great that you put it on the site when it is proven to work.

Hmmm, we should combine the backends. Add a json subaction, while i keep my own low-data format. This way the notifiers will use only one server side resource. Very cool done though :)
Thanks! Yeah, that would be really good.

Actually that would be a good idea, so that we don't have to install two SMF mods for them (which is annoying due to mod conflicts).
As I said, I was not using or requiring any kind of mod. But if Cumred and I merge theres one less problem to think of.
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc
  • Consoles, mobile devices and vintage computers owned: Hewlett-Packard 100LX, Original Commodore 64C, Tektronix AWG2021, IBM X60s, IBM X60t, Nintendo DS Lite, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, Palm m5

Online Juju

  • aka Yuki Kagayaki aka J̵̭͕͇ù̞̭̝̯̦j̴̭̙̗͖͡ù͏͓̲̕
  • Featured Project Author
  • Super User
  • Server Maintenance
  • Moderator
  • Forum Maintenance
  • Original 5
  • CodeWalrus Supporter
  • *
  • Join Date: Nov 2014
  • Location: Inside a walrus
  • Posts: 2823
  • Post Rating Ratio: +18/-1
  • Couch potato
    • jul.savard
    • juju2143
    • @juju2143
    • juju2143
    • @julosoft
    • juju-kun
    • /u/juju2143
    • juju2143
    • View Profile
    • Juju's shed
  • Gender: Female
  • WalriiPoints: 99999
Re: Pollwalrus API and Desktop Post Notifier
« Reply #5 on: May 11, 2015, 10:31:40 pm »
Speaking of combining the backends, if you guys want to use socket.io, I could always hack an interface into Walrusirc.
  • Calculators owned: TI-83+ (dead?), Casio Prizm (also dead???)
YUKI-CHAAAANNNN
In the beginning there was walrii. In the end there will be walrii. All hail our supreme leader :walrii: --Snektron

????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????:trollface:???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

Offline DarkestEx

  • Super User
  • Join Date: Jan 2015
  • Location: Germany
  • Posts: 1318
  • Post Rating Ratio: +10/-2
    • @0xbmuessig
    • @muessigb
    • View Profile
    • My homepage
  • Gender: Male
Re: Pollwalrus API and Desktop Post Notifier
« Reply #6 on: May 11, 2015, 11:05:44 pm »
Speaking of combining the backends, if you guys want to use socket.io, I could always hack an interface into Walrusirc.
I think it is a great idea, but we should have 3 protocols in this plugin:
1st: Cumreds custom protocoll
2nd: My JSON/gzip protocoll
3rd: The socket.io event protocoll
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc
  • Consoles, mobile devices and vintage computers owned: Hewlett-Packard 100LX, Original Commodore 64C, Tektronix AWG2021, IBM X60s, IBM X60t, Nintendo DS Lite, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, Palm m5

Offline Snektron

  • Lvl 69 Russian Snake
  • CodeWalrus Staff
  • Super User
  • Topic Management
  • Join Date: Dec 2014
  • Location: Netherlands
  • Posts: 3155
  • Post Rating Ratio: +31/-0
  • SSSssssss.....
    • RobinDeWalvis
    • Kzyrox
    • RobinDeWalvis
    • View Profile
    • quantuminfinity
  • Gender: Male
Re: Pollwalrus API and Desktop Post Notifier
« Reply #7 on: May 12, 2015, 05:36:21 am »
The question is: are keepalive packets lighter than http requests (with keepalive like every 5 sec and http like every minute)
  • Calculators owned: TI-84+
Legends say if you spam more than DJ Omnimaga, you will become a walrus...


Online DJ Omnimaga

  • Omni founder & CW co-founder
  • CodeWalrus Staff
  • Super User
  • Forum Maintenance
  • Original 5
  • CodeWalrus Supporter
  • *
  • Topic Management
  • Join Date: Nov 2014
  • Location: Quebec, Canada
  • Posts: 18179
  • Post Rating Ratio: +91/-4
    • dj_omnimaga
    • DJOmnimaga.music
    • @DJOmnimaga
    • dj_omnimaga
    • @DJOmnimaga
    • /u/DJ_Omnimaga
    • DJOmnimaga
    • 112/11286
    • @djomnimaga
    • @DJOmnimaga
    • View Profile
    • DJ Omnimaga music store
  • Gender: Male
Re: Pollwalrus API and Desktop Post Notifier
« Reply #8 on: May 14, 2015, 05:30:26 am »
In any case, when you start testing this on this website, check the board stats page at http://codewalr.us/index.php?action=stats and check if the page views seem to increase immediately after the post notifier does his things. If that is the case, try to ensure that users cannot set it to check more than every 60 seconds. Also yeah having every package combined would be nice, providing that they're easy to update, especially if they ever stopped working in newer OSes or firmwares.
  • Calculators owned: TI-73, TI-80 (broken), TI-81, TI-82, TI-83, TI-83+ (broken), TI-83+ (broken), TI-83+SE (broken), TI-84+, TI-84+CSE, TI-84+CE, TI-85, TI-86, TI-89T, TI-92, TI-Nspire, TI-Nspire CX (semi-broken), HP 39gII, HP Prime, Casio fx-7000G, fx-7400G+, fx-7700GE, fx-9750G+, fx-9750GII, fx-9860G, cfx-9850G, FX-1.0+, fx-CG10, fx-CP400
  • Consoles, mobile devices and vintage computers owned: Samsung i5510, Nexus 5, Atari 2600, Lynx, SMS, Game Gear, Genesis, Dreamcast, NES, SNES, N64, GCN, Wii, Wii U, GBA, DS, 3DS, PS2, PS3, PS4, PSP, PSVita, XBox 360, XBOne

Bandcamp|Reverbnation|Facebook|Youtube|Twitter
Retired Omnimaga admin (2001-11) and editor (2012-14)
??? ??? ??? ???

Offline Snektron

  • Lvl 69 Russian Snake
  • CodeWalrus Staff
  • Super User
  • Topic Management
  • Join Date: Dec 2014
  • Location: Netherlands
  • Posts: 3155
  • Post Rating Ratio: +31/-0
  • SSSssssss.....
    • RobinDeWalvis
    • Kzyrox
    • RobinDeWalvis
    • View Profile
    • quantuminfinity
  • Gender: Male
Re: Pollwalrus API and Desktop Post Notifier
« Reply #9 on: May 14, 2015, 08:37:10 am »
Ah yes, i will also check the total page views on my test forum and see if it goes up :)
  • Calculators owned: TI-84+
Legends say if you spam more than DJ Omnimaga, you will become a walrus...


Online DJ Omnimaga

  • Omni founder & CW co-founder
  • CodeWalrus Staff
  • Super User
  • Forum Maintenance
  • Original 5
  • CodeWalrus Supporter
  • *
  • Topic Management
  • Join Date: Nov 2014
  • Location: Quebec, Canada
  • Posts: 18179
  • Post Rating Ratio: +91/-4
    • dj_omnimaga
    • DJOmnimaga.music
    • @DJOmnimaga
    • dj_omnimaga
    • @DJOmnimaga
    • /u/DJ_Omnimaga
    • DJOmnimaga
    • 112/11286
    • @djomnimaga
    • @DJOmnimaga
    • View Profile
    • DJ Omnimaga music store
  • Gender: Male
Re: Pollwalrus API and Desktop Post Notifier
« Reply #10 on: May 14, 2015, 06:40:12 pm »
Actually it might be better to do it on your test board indeed. I didn't really think about it lol. On CW the issue is that there are more users online at once  so the view count is almost guaranteed to have increased by 1 before you even refreshed the page, which might make it hard to check if the post notifier increased the view count.

  • Calculators owned: TI-73, TI-80 (broken), TI-81, TI-82, TI-83, TI-83+ (broken), TI-83+ (broken), TI-83+SE (broken), TI-84+, TI-84+CSE, TI-84+CE, TI-85, TI-86, TI-89T, TI-92, TI-Nspire, TI-Nspire CX (semi-broken), HP 39gII, HP Prime, Casio fx-7000G, fx-7400G+, fx-7700GE, fx-9750G+, fx-9750GII, fx-9860G, cfx-9850G, FX-1.0+, fx-CG10, fx-CP400
  • Consoles, mobile devices and vintage computers owned: Samsung i5510, Nexus 5, Atari 2600, Lynx, SMS, Game Gear, Genesis, Dreamcast, NES, SNES, N64, GCN, Wii, Wii U, GBA, DS, 3DS, PS2, PS3, PS4, PSP, PSVita, XBox 360, XBOne

Bandcamp|Reverbnation|Facebook|Youtube|Twitter
Retired Omnimaga admin (2001-11) and editor (2012-14)
??? ??? ??? ???

 


You can also use the following HTML or bulletin board code to share it on your page or forum signature!


Also do not forget to check our affiliates below.
Planet Casio TI-Planet Calc.news BroniesQC BosaikNet Velocity Games