First 250 days of software dev - Part 18
12 min read
Day 171
Today I wrote the functions I need to use most on the front end (on a simple messaging application). It's still at the most basic level, but it includes the front end part that I need for instant two-way communication using socket.io. Tomorrow and for the rest of this week I will work on the communication between server and client. My goal is to finish the Websocket part on Saturday. I was planning to work this evening and complete the missing parts, but I didn't have the opportunity (as a result of time allocated for family and friends). Other than that, I think I can do the task. I will be very happy for myself if I can complete my requirements this week. Tomorrow I will directly adapt the simple chat application example in socket.io to the system I wrote and try to develop it from the very basics. I think I am progressing in a systematic way (a systematic progress in my own way, of course). I think I will complete the basic parts and solve the problems as I encounter deficiencies.Day 172
It took us Saturday and Monday to get the simplest view of the front end. Today it was about sending the message written in the client to the server using socket.io and the server sending a response back. Yes, simple messaging was provided. But these messages had to be stored in a persistent place like a text file or a database. In the afternoon I concentrated on this and thought about what I could do. First of all, in order to make this application work consistently, I need to have 3 users. 3 users can communicate in pairs or form a group together. I haven't yet considered the possibility of 2 people being in a group, but since a group naturally has a group name, you can tell if there is a group by the presence of the group name instead of the number of people. I thought that if there was a bot user, users could communicate with him/her. I have named this bot "Assistant" for now. The bot user could create surveys and ask users to fill them out, for example. Or when users encounter a problem, they can contact him directly. I even wondered what it would be like if users could anonymously tell Assistant about something they are afraid to say but want to raise.The most thought-provoking topic of today was how data structures should look like. At this stage, instead of using a database, the data is simply kept in json/text files. For example, "dbClient.json" is used for the user table, "chat.json" for the chat data table and "session.json" for the session. Today we thought about how to store user data and chat data (chat participants, chat id, chat contents, chat dates, etc.). The connection between these two different data structures (or tables/files) could be based on the id of the users and the id of the chats they are in. The current user id was obtained from session information.
In addition, today messages were dynamically generated using DOM methods. But the creation of messages didn't make much sense as mentioned above. Because when the user refreshes the page, all messages are naturally deleted.
Things I think I should work on tomorrow:
- If the user has a chat history, data about their past chats will be transferred to the client via socket.io. (Here, instead of transferring all messages, you can also transfer a certain character length of the last message. Just like skype's recent chats section. Then, if the user clicks on that chat, all messages can be loaded, or recent messages can be loaded, and if the user wants to see much older messages, then those messages can be pulled from the database. For now, instead of thinking about the most optimized solution, I think I will do the case where all messages are retrieved from the database)).
- If the user has no chat history, a chat will be started with the Assistant. It will be added to the database first and then shown to the client.
- If the user has a chat history and any of those chats have multiple participants, that chat will be seen as a group and the data will be passed to the client that way.
- When all this is realized, it is likely that the shortcomings on the front end will be very noticeable and they will simply be fixed.
- Another important issue is the cache structure, which I don't know much about yet. The user's past messages will definitely be retrieved from a database (or any persistent storage - like the text/json file used now). But in case the user is instant messaging, this data will of course be written to the database, but the client side data will not be retrieved from the database, but only processed by the server. When a new message is written, I guess the server does two things. First, it adds this new message to the database asynchronously. Secondly, it delivers this data to the client.
Day 173
Honestly, I feel that I am progressing in the way I want. Of course, I don't know how important my own wishes are, but I am at least a little satisfied with the work I do myself. Today, I started to dynamically add data to the statically existing front end by taking the data from the text file. I am trying to write a code that can be changed as much as I can, that is compatible, that can be corrected by a person who looks at the code at a future time by absorbing Bruce Lee's sentence "Be like water, my friend, take the shape of the container you are in, etc.". In fact, there is still nothing about OOP in the code I wrote. Just because I think I have grasped a little bit of what is meant there, the thought of "you should write a modular code Buğra" dominates in my mind. It hasn't happened yet, but I hope I will.I just came across something very small, and it was a lot of fun to learn it. I was trying to see what I could do in forEach to make the loop stop where I want it to (without affecting later parts of the code), and I found out that with the every() function, I can simply do that.
Today I only worked on presenting chat related data in a text file to the client as it should be. I should have used technologies like ajax/fetch to communicate with the backend from an html file. But since I created the instant messaging with socket.io, I preferred socket.io instead of fetch/ajax in the other part. I don't know how correct it is yet. The basic parts are still not finished yet. Tomorrow or Friday at the latest, I will try to build a structure where groups and people are there, and instant messages come as desired. There are many things missing. I have a lot more to do. I'm glad I do and I'm learning, because it's extremely enjoyable (not in a distorted hedonistic way, of course). But time continues to flow like water.
I don't know if I am doing the work of 5 people, but I think I am getting faster than before. Or I think I reinforced what you showed me a little bit (not completely, of course, but at least I think I smelled a few things like cars smelling gasoline due to high gasoline prices: how to write a good code, the technologies and tools I will use. I used to be more or less a logical person, but now I think my horizons have broadened even more. I often feel that something is not right in the parts where I am wrong. Sometimes I continue to do it wrong on purpose because the trade-off I make with my inner voice is to make a mistake and then fix that part and complete different parts). I can't do the work of 5 people, because I am one of them. I'm just a simple person with an average intelligence who wants to improve. There will always be people who are better than me in almost everything. But I will do my best.
Day 174
I can take the messages written in the past, existing chats, necessary user information (photo, name, status, etc.) and write them in the frontend. I can also show the messages written instantly in the front end as I want. But I have some very serious problems because I built the system a little wrong. I think I know how to solve it. My problem is this: I can print past messages easily. I save the instant messages to a text file asynchronously and print them to the screen at the same time. But when I enter a different chat, these instant messages disappear. They come back when I refresh the page, of course (it does it from the text file). I know why it disappears because I wanted it to disappear and I wrote a few lines of code accordingly, because if it didn't disappear, then all the messages would be mixed in all the chats. A solution I created for one problem created a problem in a different place. So I can solve the problem by keeping the messages in a main element with the chat id and adding each new message to this chat id. I show the relevant element of the active chat and hide the part related to other chats. I don't think the way I'm going to add each new message to the database and then not pull it from there is not wrong and can be implemented. But I will fix my problems, I have completed what I wanted to do today, but I haven't solved the problems afterwards.Tomorrow, if I implement the solution I mentioned above and fix the problem, I will get a bit more specific. I need to show the status that one of the people currently in the chat is writing a message. I need to give the user the ability to edit, delete, etc. Then I need to allow the user to change their status (available, busy, away, offline, etc.). I need to allow the user to go offline when they log out or disconnected. I need to show the user new incoming messages with a notification.
I'm trying to take firm steps forward.
Day 175
Only one user is in SSO. Fictitious users and chat histories are retrieved from a text file. As it should be, all old and new messages and chats are shown in the front end. Two important problems were solved today. As you said, the application was exhibiting abnormal behavior when accessing the same url from different tabs (or browser) because it was broadcasting to all sockets. I fixed that, and then I completed the status change (there are still a few things missing, but I will come back after I complete the more important things).There are 3 parts left, such as reporting the presence of unread messages, showing who is typing in the chat with "is typing", and sending files. For the first two parts, I need to send messages from different users. For this, I need to edit the SSO section again, because in order for users to log in and write, they need to be registered in SSO and I need to transfer the necessary user data from SSO to the client server. Tomorrow I will add in the SSO section the possibility to have different users to debug the application (as it should be. I should have done it in the first place, but I think I thought too superficially when I said that I shouldn't overengineer while evaluating the problem). This edit may take some time but I have to do it. I don't think I'm in the dark, I can see the light, I think I'll get there. I expected myself to finish the part about the socket this week. After all the main problems are finished, I want to learn how to make voice and video calls and implement it if possible (if it is possible in many applications, why not?).
Day 176
Today I made corrections to the code I wrote in the past. It's not finished yet, so I guess I'll look at it for a couple of hours tomorrow evening. I keep the authentication information of multiple users in SSO now, but I also need to edit the session part of the client. I guess I will complete it on Monday, and if I do, I think there will be no major problem left.Day 177
Even though I fixed the SSO side, I should have made the session to support more than one session in the backend part that uses SSO. I tried to complete this situation today. I set up a structure that seemed to work correctly by following a wrong path. But then, while I was editing this structure to work correctly, I produced results that I did not want. But I think I will complete that part tomorrow. If I complete it tomorrow, a big workload will be completed. I believe it is about to happen.Day 178
I solved the problem I encountered yesterday and could not solve it this morning with a calm mind. The problem was to keep data of more than one user in the session. Since SSO was single-user until now, I was statically creating and simulating multi-user processes on the client server myself. I solved this problem this morning. Then I correctly created the problematic structure in the status change, the "is typing" part. Towards the end of the day I looked at the notification issue, but for now I don't keep the notifications in any persistent file. Temporarily only the users who are online can see the notifications that are currently coming. I'll come back to this later, because for now it's time for another problem: uploading files using socket.io. I need to send documents, photos, videos, and audio files to the server and save them in a permanent folder. On the front end, I need to provide these files to the client of course. I've often come across 2 different node modules for file uploading. I'm either going to build it completely myself or use one of these modules. I'm going to try first without using any 3rd party modules.Of course, the nature of things is that problems never end and there is always a solution to these problems.
Day 179
I got into uploading files today. At first I thought it was hard, but when I looked into it I realized it's easy. It's just a bit tricky. Until now, the messages sent were only shown to the user as text. I hadn't thought about how to send, save and display photos, videos, documents (with pdf, doc extensions), audio files to the user. I started that part today.The process of instantly sending the file to the server and forwarding the file to the corresponding chat is complete. I just need to organize the different situations for different file types a bit more. I haven't added the part about the file being sent to the chat history yet. Tomorrow I will finish these things.
I guess I'll finish most of it tomorrow. The problem is that despite all the OOP work I've done, I still don't use that knowledge at all (although logically I think I'm trying to keep the modularity high, it's very insufficient). I think I should have made the diagrams at the beginning of the project and acted accordingly. Anyway, this was a new experience and gain for me.
Day 180
Total 180 working days reached. A total of 7 months have been completed (7 months - 53 minutes). Today, with socket.io, a file that can have about 10-15 different types (mime types) was saved in a local folder and while saving, the file was sent to the people in the relevant chat at the same time. Of course, it is necessary to keep the data about the files sent in the past chat information while opening the application. This part was started this evening. The goal is to finish it by lunch tomorrow. Actually, the logic is very simple but the implementation will take some time. When I think about it, there are still a lot of mistakes and shortcomings. The notification part is not completed, chats have been added statically (At this stage, all users can only chat with Assistant by default if they don't have a chat). Tomorrow, the necessary work will be done to show the user the files uploaded in the past. Then we will investigate how to record audio data using a microphone and send it to the other user. This will be done. Afterwards, live voice and video conversation will be added to the system. Although the dreams seem like Paris, it is estimated that the project will be completed in the desired way (like customized Skype).Today, documents such as zip, rar, pdf, audio and video files were uploaded to the system. They could be uploaded. It was also transmitted live to the client and the client was able to download the files presented as documents. Photo, video and audio files could be opened and viewed in the front-end section and also downloaded. There was a problem today. Socket.io by default allowed sending a maximum of 1Mb of data. If we wanted to send more, it would close and reopen the connection as if nothing happened.
An option for the maximum size of the file that can be sent was found in the socket.io documentation and the maximum size of the file that can be sent is now 100 Mb. It is not yet known (I don't know) what kind of problems this can cause. Sending the data in chunks was initially considered, but this shortcut was chosen because it seemed like a more laborious way.
I think it's happening. I feel like with order comes chaos, and I'm going to organize it as much as I can.