Join us on Discord!
You can help CodeWalrus stay online by donating here.

Pollwalrus API and Desktop Post Notifier

Started by DarkestEx, May 11, 2015, 09:24:04 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

DarkestEx

Hello :walrii: sses!
I am working on a convenient API and desktop notifier for post replies.
The API is currently located here: http://api.muessigb.net/walrus_notify.php.
It outputs data using minified json and reads a number of get parameters over http.
If the client supports it, the data is sent using gzip compression.
The API is publicly usable for anybody who wants to make a post notifier for CW and should really not be overused (normal polling every minute is okay), but I would really like you to download it instead and upload it to your own server if you go below this refresh rate.
The API does alot of things: It downloads and parses the feed (saving you the trouble to configure the feed and saves you some traffic), makes the data more machine readable, converts the relative dates to unix timestamps, converts the relative links to absolute ones, parses the HTML and does one of three things (mode is selectable over get parameter):  1: Simply strip all html;  2: Strip all html, but preserve convert smileys;  3: Strip all html but convert smileys and convert tags like b and i to ** and // markup;  4: Leave the html intact
The api makes polling easier and you can ask the API if any updates happened since a specific post id. Also you can specify how many posts are checked or returned.

The desktop notifier will be using this API and refreshes between twice a minute (only for me), every minute or every 15 minutes.
It is going to get a similar design to the one of @Cumred_Snektron

More details will follow.
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc, Hewlett-Packard 100LX, Hewlett-Packard 95LX
  • Consoles, mobile devices and vintage computers owned: Original Commodore 64C, C64 DTV, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, PlayStation 2

Dream of Omnimaga

Does this fetch the site RSS feed every 1-15 minutes? Because everytime the RSS feed is open, it increases the board stats view count by 1, so if your tool opens it instead of using an SMF mod like Cumred's post notifier, then a refresh every minute will inflate our site view count by 1440 everyday. Not that this would be a serious issue, but it would kinda make our site traffic stats look less legit.

However, if by same design as Cumred's, you meant that it won't do that, then ignore my post. :P I think a Windows or Linux post notifier would be nice, in addition to the Android one. I will most likely add download links to them on http://codewalr.us/index.php?page=recent2 if they are hosted on our server and fully reliable, useability and resource-usage-wise.
  • Calculators owned: TI-82 Advanced Edition Python TI-84+ TI-84+CSE TI-84+CE TI-84+CEP TI-86 TI-89T cfx-9940GT fx-7400G+ fx 1.0+ fx-9750G+ fx-9860G fx-CG10 HP 49g+ HP 39g+ HP 39gs (bricked) HP 39gII HP Prime G1 HP Prime G2 Sharp EL-9600C
  • Consoles, mobile devices and vintage computers owned: Huawei P30 Lite, Moto G 5G, Nintendo 64 (broken), Playstation, Wii U

Snektron

Hmmm, we should combine the backends. Add a json subaction, while i keep my own low-data format. This way the notifiers will use only one server side resource. Very cool done though :)
  • Calculators owned: TI-84+
Legends say if you spam more than DJ Omnimaga, you will become a walrus...


Dream of Omnimaga

Actually that would be a good idea, so that we don't have to install two SMF mods for them (which is annoying due to mod conflicts).
  • Calculators owned: TI-82 Advanced Edition Python TI-84+ TI-84+CSE TI-84+CE TI-84+CEP TI-86 TI-89T cfx-9940GT fx-7400G+ fx 1.0+ fx-9750G+ fx-9860G fx-CG10 HP 49g+ HP 39g+ HP 39gs (bricked) HP 39gII HP Prime G1 HP Prime G2 Sharp EL-9600C
  • Consoles, mobile devices and vintage computers owned: Huawei P30 Lite, Moto G 5G, Nintendo 64 (broken), Playstation, Wii U

DarkestEx

Quote from: DJ Omnimaga on May 11, 2015, 10:03:38 PM
Does this fetch the site RSS feed every 1-15 minutes? Because everytime the RSS feed is open, it increases the board stats view count by 1, so if your tool opens it instead of using an SMF mod like Cumred's post notifier, then a refresh every minute will inflate our site view count by 1440 everyday. Not that this would be a serious issue, but it would kinda make our site traffic stats look less legit.

However, if by same design as Cumred's, you meant that it won't do that, then ignore my post. :P I think a Windows or Linux post notifier would be nice, in addition to the Android one. I will most likely add download links to them on http://codewalr.us/index.php?page=recent2 if they are hosted on our server and fully reliable, useability and resource-usage-wise.
Currently I am using the SMF RSS feed and alot of PHP ninja skillz to make it work. The feed is fetched on every API request and then processed on my server (produces quite some load due to gzip compression). The polling can be done in any possible rate. Also I was planning to cache the response unless thats overridden using a get parameter.
Great that you put it on the site when it is proven to work.

Quote from: Cumred_Snektron on May 11, 2015, 10:17:18 PM
Hmmm, we should combine the backends. Add a json subaction, while i keep my own low-data format. This way the notifiers will use only one server side resource. Very cool done though :)
Thanks! Yeah, that would be really good.

Quote from: DJ Omnimaga on May 11, 2015, 10:18:35 PM
Actually that would be a good idea, so that we don't have to install two SMF mods for them (which is annoying due to mod conflicts).
As I said, I was not using or requiring any kind of mod. But if Cumred and I merge theres one less problem to think of.
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc, Hewlett-Packard 100LX, Hewlett-Packard 95LX
  • Consoles, mobile devices and vintage computers owned: Original Commodore 64C, C64 DTV, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, PlayStation 2

Yuki

Speaking of combining the backends, if you guys want to use socket.io, I could always hack an interface into Walrusirc.
  • Calculators owned: TI-83+ (dead?), Casio Prizm (also dead???)
  • Consoles, mobile devices and vintage computers owned: A lot
Read Zarmina!
YUKI-CHAAAANNNN
In the beginning there was walrii. In the end there will be walrii. All hail our supreme leader :walrii: --Snektron

if you wanna throw money at me and/or CodeWalrus monthly it's here

DarkestEx

Quote from: Juju on May 11, 2015, 10:31:40 PM
Speaking of combining the backends, if you guys want to use socket.io, I could always hack an interface into Walrusirc.
I think it is a great idea, but we should have 3 protocols in this plugin:
1st: Cumreds custom protocoll
2nd: My JSON/gzip protocoll
3rd: The socket.io event protocoll
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc, Hewlett-Packard 100LX, Hewlett-Packard 95LX
  • Consoles, mobile devices and vintage computers owned: Original Commodore 64C, C64 DTV, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, PlayStation 2

Snektron

The question is: are keepalive packets lighter than http requests (with keepalive like every 5 sec and http like every minute)
  • Calculators owned: TI-84+
Legends say if you spam more than DJ Omnimaga, you will become a walrus...


Dream of Omnimaga

In any case, when you start testing this on this website, check the board stats page at http://codewalr.us/index.php?action=stats and check if the page views seem to increase immediately after the post notifier does his things. If that is the case, try to ensure that users cannot set it to check more than every 60 seconds. Also yeah having every package combined would be nice, providing that they're easy to update, especially if they ever stopped working in newer OSes or firmwares.
  • Calculators owned: TI-82 Advanced Edition Python TI-84+ TI-84+CSE TI-84+CE TI-84+CEP TI-86 TI-89T cfx-9940GT fx-7400G+ fx 1.0+ fx-9750G+ fx-9860G fx-CG10 HP 49g+ HP 39g+ HP 39gs (bricked) HP 39gII HP Prime G1 HP Prime G2 Sharp EL-9600C
  • Consoles, mobile devices and vintage computers owned: Huawei P30 Lite, Moto G 5G, Nintendo 64 (broken), Playstation, Wii U

Snektron

Ah yes, i will also check the total page views on my test forum and see if it goes up :)
  • Calculators owned: TI-84+
Legends say if you spam more than DJ Omnimaga, you will become a walrus...


Dream of Omnimaga

Actually it might be better to do it on your test board indeed. I didn't really think about it lol. On CW the issue is that there are more users online at once  so the view count is almost guaranteed to have increased by 1 before you even refreshed the page, which might make it hard to check if the post notifier increased the view count.

  • Calculators owned: TI-82 Advanced Edition Python TI-84+ TI-84+CSE TI-84+CE TI-84+CEP TI-86 TI-89T cfx-9940GT fx-7400G+ fx 1.0+ fx-9750G+ fx-9860G fx-CG10 HP 49g+ HP 39g+ HP 39gs (bricked) HP 39gII HP Prime G1 HP Prime G2 Sharp EL-9600C
  • Consoles, mobile devices and vintage computers owned: Huawei P30 Lite, Moto G 5G, Nintendo 64 (broken), Playstation, Wii U

Powered by EzPortal