“An idea that makes someone’s life better.”
Today I am starting this small project that might become bigger project at some point. This idea came up once I found there is a need to automatically detect a new blog posting in my wife’s account and make a new posting in Medium account with corresponding contents, which is basically aimed at increasing the number of visitors and readers of my wife’s blog contents.
So, how am I going to implement this?
first off, I began to write out some high level business requirements before actually getting down to writing codes. This process take up much of time in reality rather than writing down codes indeed.
Requirements
- Automatically crawl necessary contents data from Naver Blogs.
- Save the contents data to some persistent layer so that other backend process can use to post in Medium.
- Preprocess the extracted data in favor of Medium API Request Body schema.
- Post the saved contents in Medium in the same order of contents as in original contents.
- Save the state of crawled posts in persistent layer so for programs not to duplicately post in Medium platform.
These requirements will be a baseline for any further business process unit functionality.
and then I came to think what kinds of technologies I am going to use to implement the requirements.
Tech Stacks
it’s quite normal to think about what sort of things I should use to make a program. and I would like to use a stack of techs as belows :
- Python for language — a light-weigh programming language to start with abundant eco system for programming. In particular, for this project, I am familiar with Scrapy library. This open source is quite strong in crawling and parsing HTML data. and for dependency management, I can simply leverage the power of pipenv with simple command lines.
- Pycharm for IDE — I am used to Jet Brains’s IDE. and it is the most comfortable one for me as a python programmer like I am used to IntelliJ for Java programming.
- POSTMAN for API — To get familiar with Medium API and understand what kinds of parameters I should provide for the actual programs, I mostly enjoy using postman.
Thus, the 3 techs are going to be mainly used to actually make it work.
What needs to be done next?
So far, I have taken very very high level approach to start a project. and the next thing to do is definitely the Proof of Concept.
I believe the idea can only become an tangible outcome when the concept is proved to be doable, tangible, feasible and implementable.
TO BE CONTINUED…
