Personal Notification Hub
A long time back one of my friends asked me to check this cool idea which was simply a small display device that “keeps you updated for anything important for you”. Somehow the idea did not succeed in raising funds and eventually scraped out.
The idea
Though it was not possible for me to create hardware for such an idea(feel free to contact me if you have that level of skill) but I tried to achieve the software part of this idea. What I tried to create was a centralized hub that provides me unread notifications count that I have on different social platforms(WhatsApp, Skype, Facebook, Youtube). The hub will just provide unread notification counts and not the unread messages because the idea is just to have a crisp summary of the things and not a detailed view. You don't have to open up your phone to check if you are missing a message. If you noticed a count, you always have the mobile phone to go and see the message.
Currently available options in the market
When I tried googling, I didn't come across any utility or application which serves this purpose. So that gave me another reason to create this utility. The first idea that came to my mind was that I would hit web service let’s say for Facebook and get a notification count. Similarly, I would then hit web service for WhatsApp and get my count. Unfortunately, there are no such services available for these platforms(Facebook used to have one but now is discontinued). Slack is pretty good at providing such useful web services for itself and I think other social platforms should seriously consider this. Moreover, it might be a very tedious task to get a good understanding of web services provided by each social media platform, and to add to that they might be paid or have some restrictions on hits/day.
Implementation
When nothing comes in handy you always have an option for web scraping. Yes, and reading the word scraping most of you would already figure out what is coming up next in this article. Yes, we will use the Selenium Python framework, open up parallel browser instances for each social media platform and do the scraping. To add to this utility we will create a Flask-based microservice so that you hit the service and it returns the counts for unread notifications for each social media platform. I have personally done the implementation on the Windows system but you can give it a try if the same works on Linux or Raspberry systems(they may not support automation tools in head mode).
- To begin with, you would need Chromedriver for Windows that can be downloaded from here. Any version will work.
- Import necessary packages: selenium for scraping purpose, threading for creating parallel threads for each social media platform, and flask for creating the final microservice.
from selenium import webdriver
import time
import threading
from flask import Flask, request, jsonify, make_response3. Create the driver instance for each social media platform
fbDriver = webdriver.Chrome("chromedriver")
fbDriver.get("https://www.facebook.com/")
waDriver = webdriver.Chrome("chromedriver")
waDriver.get("https://web.whatsapp.com/")
ytDriver = webdriver.Chrome("chromedriver")
ytDriver.get("https://www.youtube.com/")4. Define global variables that hold the counts for each SoMe(Social media) platform. Why global? So that the scraping(setter of the variable) and Flask-microservice(getter of the variable for the end-user) use the same variable and there is no discrepancy in values.
fbNotifications=0 #for facebook related notification
msgNotifications=0 #for messenger related notification
waNotifications=0 #for whatsapp related notification
ytNotifications=0 # for youtube related notiifcation5. Create a handler function for each social media platform where the real scraping stuff happens. Update the global variable inside the function. Here is an example of a scraping function for Facebook
def fbCountManager():
global fbNotifications
global msgNotifications
while True:
try:
fbNotifications=fbDriver.find_element_by_id('notificationsCountValue')
msgNotifications=fbDriver.find_element_by_id('mercurymessagesCountValue')
except:
fbNotifications=0
msgNotifications=0
fbDriver.refresh()
time.sleep(20)
What we want is the values of these 2 badges from UI(the first badge gives notification count for messenger and the second one is for Facebook notification). I would not be discussing how I identified the element id’s but you can go through Selenium syntaxes and tutorials to check more about that.
Here is a similar function for whatsapp
def waCountManager():
global waNotifications
while True:
try:
waNotifications=0
allMsgs=waDriver.find_elements_by_css_selector('.OUeyt')
for msg in allMsgs:
waNotifications=waNotifications+int(msg.text)
except:
waNotifications=0
waDriver.refresh()
time.sleep(20)Note that we are not storing any credentials for any platform in our python code because of security concerns. So you would have to perform login to each platform manually one time. Though this login process can also be automated through Selenium but I recommend not to compromise your credentials in a plain script file.
6. You can similarly create scraping functions for each social media platform. Next, you have to create and start parallel threads for each platform. This is how you do it. The target parameter takes in the function that this thread should be calling.
fbThread = threading.Thread(target=fbCountManager)
fbThread.start()
waThread = threading.Thread(target=waCountManager)
waThread.start()
ytThread = threading.Thread(target=ytCountManager)
ytThread.start()7. The final part is creating a Flask-based microservice so that the end-user(which is YOU) hits the service and gets the accumulated response containing counts for each social media platform. I really like the concept of microservice(the user doesn't care how that data is getting fetched, he just hits the service and gets the results. he is happy)
app=Flask(__name__)
@app.route('/getcounts/', methods=['GET'])
def respond():
response_body = {
"fbNotifications":fbNotifications,
"msgNotifications": msgNotifications,
"waNotifications": waNotifications,
"ytNotifications": ytNotifications
}
res = make_response(jsonify(response_body), 200)
return res
@app.route('/')
def index():
getCoronaCounts()
return '<h1>Notiifcation App</h1>'
if __name__ == '__main__':
# Threaded option to enable multiple instances for multiple user access support
app.run(threaded=True, port=5000)Notice how we are using the global variables that we had earlier defined. I know I could have done better here and added some security or some more fancy features. But the goal was to create a quick solution. But now that you are aware of the idea, you always have scope for improvement.
8. Everything is in place, let's try to test what is response this microservice is returning. The service is running on port 5000 which we defined in the last line.
What next
If you have completed creating the utility or hub, you have the following ideas to use this utility(I haven't done any of these myself, so I would be pleased if you share them with me)
- Try to deploy it on Raspberry Pi so that you are not relying on your personal laptop the entire day to run this.
- Create the notification count service as public using ngrok, so that you can access the counts outside your home network.
- Use it in your smart mirror or background wall displays.
Terms of Usage
I would clearly like to mention that this utility or hub is just for your personal use. As you have clearly seen that there is no added or extra security present, so use it very carefully. Also, refer to the usage terms of the site which you are scraping, it is not legal to perform any kind of scraping on some websites.
