Berkeley has released a nice introduction to numerical methods using Python, “Python Programming And Numerical Methods: A Guide For Engineers And Scientists”. If you are looking for a book that will get you up to speed quickly on Python but also serves as a reference for more advanced topics this book will work well for you. Engineers and scientists will find this reference handy. The second half of the book reviews linear algebra and matrix manipulation in Python and walks through all the numerical methods I learned at university, plus a few more.
The code is available on Github and the book is available as a .pdf download or as an interactive Jupyter notebook.
I purchased a few of the new Raspberry Pi Pico’s from Adafruit recently. I’ve been playing with MicroPython, and CircuitPython. I ran across a nice tutorial at Tom’s Hardware that walked through simple music output using the Pico with a passive buzzer. I liked the idea so I modified the code to also take a list of note durations. This makes the “music” a little closer to the original.
We recently moved a data repository from MSSQL to Postgresql. The canonical data is stored in Oracle 10g and encoded as UTF-8. MSSQL was storing in LATIN1, unbeknownst to me. It worked somehow. Postgres was setup UTF-8. The migration was simple and all python based reports worked with a simple update to the database connection string. Excel reports began throwing errors on import. The errors were related to UTF-8 characters not being supported by WIN1252 (cp1252) encoding.
After searching there were a few possible solutions.
Use UNICODE postgres drivers instead of ANSI
Send data to .csv files first and then import to Excel with UTF-8 encoding turned on.
Update the encoding of the local repository.
Option 1 was the easiest. We tried this but found that regardless of the driver support, Excel (Office 365) would not properly import certain UTF-8 characters. In particular U2265, U215b, and a hex character 0x9d. So, choice of driver did not matter.
Sending data to .csv works. However, there are additional steps necessary for this workflow. We need a cron job to write out the .csv files. Then we are storing the data twice. Excel handles updates to an existing .csv connection well but initial imports typically need work. In particular, column types must be updated. Technically savvy users handle this well but most people don’t want the hassle.
We knew that changing the encoding of the database would move the problem from Excel to the import script but that problem would be ours and be visible. We chose that route.
Changing the encoding did introduce import errors in our python script but they were easily caught and handled.
We first updated the Postgresql encoding
\encoding WIN1252
Then we added exception handling to the import threads to catch UnicodeExceptions
As was expected the issue existed in only a few tables and columns of the replicated data. Open ended text fields for notes were the main culprit. After identifying the sources and specific characters causing the problem we updated our import scripts to catch these characters before import.
The old entries were cleaned up using postgres’s REGEXP_MATCHES and REGEXP_REPLACE function. For example
SELECT REGEXP_MATCHES(my_column,regex,flag) from my_table;
UPDATE my_table
SET my_column = REGEXP_REPLACE(my_column,regex,replacement_string,flag);
In this case regex was U&’2265’, U&’215b’, and E’\x9d’.
Wet filament will ruin a print. Many plastics are hygroscopic (water absorbing). Their affinity to water differs but over time almost all will absorb water. Water absorption can cause problems in several ways
Filament will expand therefore clogging the printer more easily
Trapped water will boil off at printing temperatures. This is characterized by popping and clicking noises. This will lead to voids in the print, and increases wear on the nozzle.
Wet filaments cause stringing artifacts. They are unsightly but may also snag during the print. This could lead to a print failure.
I mainly print in PLA. It does not have a high affinity for water but due to my location, Houston, and my environment, a very old townhome with undersized AC capacity, my filaments get wet quickly.
As is my style, that was a long introduction to show off this great print that I found recently. PrusaPrinter member P.Tech developed a snap fit desiccant canister that fits perfectly inside of filament spools. Sizes are easily adjusted with a parametric OpenSCAD file. These can be used during storage to maintain a dry environment. I prefer reusable desiccant beads. These containers work perfectly. I recommend printing 5 or 10.
I spent a lot of time optimizing a python script last week. The script takes a list of open orders for a manufacturing plant. It joins this table with a list of open purchase orders and manufacturing orders. Finally, it joins this with a table of bill of materials and stock levels. The resulting dataframe has approximately 3.5 million rows and is a little over 1.0 Gb in memory. The script then checks every order line for the most effective, “fastest” as in lead time, method to fulfill the order. Lines that can be fulfilled now are updated so that order planners can send those items to the floor immediately.
The initial script used the pandas iterrows() function to loop over the data. Two loops are necessary because customer orders may be fulfilled by manufacturing orders with their own bill of material. This resulted in an average run time of just over 15 minutes. Then I found this gem, “How To Make Your Pandas Loop 71803 Times Faster”. I have not been able to take full advantage of the pandas and numpy vectorization. I suspect I can make further improvements. However, the code now runs in 6 minutes.
I also received a nice article in my RSS feed this week regarding Python f-strings. It is a quick introduction and a fast read. I use f-strings almost exclusively. If you are not familiar with them or need a few pointers check out the link.
Last year I linked to the AAScan project. AAScan is an open source 3D scanning platform based on Arduino. My 3D printing hobby has recently gone from primarily practical prints to making miniatures for my kids. My skills with digital sculpting are poor. I think it’s time to take on this project.
I have had a nice discussion with some other people that are interested in the topic as well. In addition to AAScan there is an interesting and well documented project called Scanning the Void. I have an idea to combine these two projects and improve the depth mapping using the new iPhone 12 camera system.
I think it should be possible to use the infrafred depth map to improve how the images are stitched together. It might also be possible to use the three cameras to identify common points on the object and triangulate a more accurate depth map. I haven’t tested any of this yet. It’s just some ideas. I intend to build the rotary table first. I have some spare Arduinos and steppers right now so it’s a good time to build it.
I also think the ideas from “Scanning the Void” are great. The system needs very dark backgrounds. This is the perfect application for the new ultra-black pigments that are being developed like Musou Black. If you haven’t seen Musou Black check out this YouTube video where a guy built a room and painted it with Musou Black.
🏈 It’s NFL Conference Championship weekend and the road to the SuperBowl runs through Green Bay. This is the fourth NFC Championship game for the Packers since 2014. No other NFC team has been in more than one. Unfortunately, we haven’t made the SuperBowl since 2010. Maybe today is the day. I’m excited. Packers nation is excited. Go Pack Go!
If you’re a WhatsApp user, pay attention to the changes in the privacy policy that you’re being forced to agree with.
tl;dr - Facebook is going to take your data and use it however they want.
Schneier gives links on his blog. I’ve said it before and I’ll say it here. Facebook is a criminal enterprise. I do not recommend using anything owned by them. I haven’t regretted my decision to delete WhatsApp.
Here’s a story from 3DPrint.com that ticks all my boxes.
From artwork and historic buildings to monuments and other priceless artifacts, technologies like 3D scanning and 3D printing are often called on to help restore and preserve touchstones of history. A recent example is a 193-year-old sterling silver menorah that belongs to the Mollie and Louis Kaplan Museum of Judaica at Congregation Beth Yeshurun, which houses more than 700 historical pieces dating as far back as the 17th century. Made in Russia in 1827, this menorah stands almost two feet tall, and features eight hinged oil lamps in the form of lions; unfortunately, one of the lions went missing sometime in the last thirty years.
Herman Fontenot is the Business Development Manager for 3DPrint Texas, the company that was hired to do the restoration. Fontenot played running back for the Cleveland Browns from 1985-1988 and the Green Bay Packers from 1989-1990. He played longer for the Browns but I’ll claim him.
This wasn’t a simple upload an STL and print type of job. The team had to use 3D scanning to develop a base model. They had to use 3D sculpting to add fine details. They employed 3D printing to produce high resolution molds, and they used the age old lost wax casting method to pour a new silver lion. It’s a beautiful restoration.
Since we are already considering women who make stuff let’s talk about Gwynne Shotwell. She is a mechanical engineer with a long history in the aerospace industry and she is currently the COO of SpaceX. The Autodesk Fusion 360 blog recently published a profile of her and I found it interesting.
One thing they didn’t mention is how she manages to indulge Elon Musk’s fantasy while keeping him out of the day to day operations of SpaceX. We’ve all seen what a mess he can make in a company. She’s found a way to keep him involved while not allowing him to make a mess of things. There is bound to be a useful book that could be written about the interactions between those two.
