Week 14 – Post Mortem

--Originally published at TC3045 – Sagnelli's blog

Howdy,

So last blog post I talked about my Database course project. We are using MongoDB as our backend along with Flask. We think is easier to implement python code with Flask than using exec functions on Node. Also, the learning curve of Node is way more difficult than Flask’s. Therefore, Flask is the winning framework to use. We won’t be using any agile methodology, since we started the project way too late. We are using XP methodologies to finish on time.

Stay tuned for this project’s advancements.

Week 14 – Pre Mortem

--Originally published at TC3045 – Sagnelli's blog

Howdy.

We are almost at the end of this course; only two weeks left. Luckily, we already finished this project; however, I can already feel the stress of working with the pending 3 projects for other courses. Actually, we might use this course’s project as a starter for our Advanced Database course project. We are thinking on developing a web app that based on data displays a heat map of parties popularity among different zones. We are focusing on Jalisco, and parties competing for governor, and senator position.

Stay tuned on the developments of this week other projects.

Week 7 – Post Mortem

--Originally published at TC3045 – Sagnelli's blog

Howdy partner,

As you may have read, or maybe not, in the pre mortem blog for this week, I worked on being able to mine tweets from an account since the beginning of their time. I was able to do that by creating a loop in which 200 tweets were gathered using the API of Twitter. Also it is possible to specify the people that you want to gather tweets from by writing their user names in a txt file stored in the same location.

This time we want to gather all the tweets from @RicardoAnayaC , @lopezobrador_ , and @JoseAMeadeK to create a words map. This map will allow us to know which words are most used by the candidates to measure their popularities.

Stay tuned, for the next weeks’ progress

Week 5: Post-Mortem

--Originally published at TC3045 – Sagnelli's blog

As you know we already had a streamer to use with the Twitter API for streaming tweets. With that streamer we were only able to filter using key words; however, tweets from all sources arrived. Now, added a second streamer, where we are now able to filter from whom we want our tweets. This list of users is provided via a .txt file, and we can also use a list of keywords for a more thorough filtering.

What did we do with the first streamer?

It is now fully working, adding filtered tweets to the database, and printing who tweeted the matching results.

The project’s development is going great.

Stay tuned for next weeks developments.

Update – Elections Analyzer 2018 improvements

--Originally published at TC3045 – Sagnelli's blog

This week a member of the team automatized the setup, and run methods of the application. As this is a two part project, we are focusing on the python micro-services of gathering, cleaning, and storing data in our database.

Database

We are using a JSON to normalize data for our relational MySQL database. We already discovered how to establish connection from Python to a MySQL database using the PyMySQL library to make DDL & DML queries.

This is an example of the code to do so.

from __future__ import print_function
import pymysql

conn = pymysql.connect(host='', port=, user='', passwd='', db='',autocommit=True)
cur = conn.cursor()

#cur.execute("CREATE TABLE Partidos ( ID int NOT NULL, nombre varchar(50), PRIMARY KEY(ID)); ")
cur.execute("INSERT INTO Partidos VALUES (111,'PAN','IZQ')")
cur.execute("INSERT INTO Partidos VALUES (112,'MORENA','DER')")
cur.execute("INSERT INTO Partidos VALUES (113,'PRI','IZQ')")
cur.execute("INSERT INTO Partidos VALUES (114,'MOVIMIENTO CIUDADANO','IZQ')")
cur.execute("SELECT * FROM Partidos")
cur.execute("DELETE FROM Partidos")
print()
for row in cur:
    print(row)
cur.close()
conn.close()

This is what I’ve done so far in the project, and I learned how to use micro-services in Python. I will continue doing generic automatization of queries for when the database is up and running.

Keep tuned for further news on the development of the project.

Update – Elections Analyzer 2018 improvements

--Originally published at TC3045 – Sagnelli's blog

Database

This is an example of the code to do so.

from __future__ import print_function
import pymysql

conn = pymysql.connect(host='', port=, user='', passwd='', db='',autocommit=True)
cur = conn.cursor()

#cur.execute("CREATE TABLE Partidos ( ID int NOT NULL, nombre varchar(50), PRIMARY KEY(ID)); ")
cur.execute("INSERT INTO Partidos VALUES (111,'PAN','IZQ')")
cur.execute("INSERT INTO Partidos VALUES (112,'MORENA','DER')")
cur.execute("INSERT INTO Partidos VALUES (113,'PRI','IZQ')")
cur.execute("INSERT INTO Partidos VALUES (114,'MOVIMIENTO CIUDADANO','IZQ')")
cur.execute("SELECT * FROM Partidos")
cur.execute("DELETE FROM Partidos")
print()
for row in cur:
    print(row)
cur.close()
conn.close()

This is what I’ve done so far in the project, and I learned how to use micro-services in Python. I will continue doing generic automatization of queries for when the database is up and running.

Keep tuned for further news on the development of the project.

Election year: let’s be genuine, shall we?

--Originally published at TC3045 – Sagnelli's blog

2018 is an important year for Mexico, where the next six years are supposed to be defined by the Mexican people; however, corruption has always interfered with democracy, as the government has been accused of manipulating the votes.

This is the problem we are trying to solve with our project. Now, the important question is:

Who are we?

We are a group of students between 6th and 8th semester of Computer Science at Tec de Monterrey Campus Guadalajara:

Alfonso Contreras
Arturo González
Alejandro Vázquez
Michelle Sagnelli

What is our solution?

Basically, in one sentence, we are building a series of microservices that will let us determine who is the best acclaimed, and the most popular presidential candidate according to Twitter.

How are we supposed to do it?

We will apply data mining using Python Streaming Jobs, and Twitter’s API to temporarily store tweets in JSON’s. Afterwards, this data will be shown and saved for later use.

The challenge is to clean data by mining keywords, eliminating stopwords, and assigning tokens by tweet importance. Henceforth, this “clean” data will be used to analyze with machine learning the importance of this year’s candidates, and political parties. Finally, this information will be stored in JSON format for further analysis of political parties information, and candidates’ level of acceptance.

Extras

We are trying to implement location-based analysis of tweets, and being able to find which tweets belong to bots for achieving a more successful analysis.

This should be fun. I am very interested in this project, as it is challenging, and interesting. If you are interested too, do not hesitate in contacting me, and stay tuned with mine, and my colleagues’ , future blog posts.

Regards,

Mike.

Elections time!

--Originally published at TC3045 – Alfonso Irai

It’s that time of our lives again. It’s that time where wherever we sight, we see some candidate ad or poster, and you know what? It really sucks! I think it’s a huuuge waste of money and you’re not gonna let me lie; it’s a visual pollution.

But let’s not talk about that. We’re here to talk about our project.

Our extraordinary team made up by Alejandro Vazquez, Cesar Gonzalez, Michelle Sagnelli and I, will build microservices that will let us know which is the most popular Mexico’s presidential candidate using Twitter’s social network.

We will be using data mining using Twitter’s API and Python Streaming Jobs to save tweets temporally in JSON’s.

The challenge here is that we will need to “clean up” and organize the data by separating by tokens of importance and deleting stop words, to be able to extract the data in a more objective way. Also (and really exciting), we will be using Machine learning to analyze the candidates and see the level of importance of them, taking positive and negative tweets not just for them, but for their political party.

I know, I know. There will be a lot of bots trying to make our lives impossible. And because of that, deleting bots will be an optional feature that we will develop if everything else goes on the right way.

I’m really excited to see this project done, and if you are interested in knowing how we are doing, please stand by for my future posts.

Love you, guys!