Welcome to My Blog

Using Python to establish a connection through Proxy/transport/intermediate server/(something in the middle) to your FinalDestination server

Quick and yet absolutely disgustingly insecure way to establish a password based authenticated connection to a server through proxy (aka jumphost/intermediate server/proxy).  Hopefully someone will find this useful. Blindly trust server may invoke  armageddon… Source code inspired by https://stackoverflow.com/questions/21609443/paramiko-proxycommand-fails-to-setup-socket

More on how to use the client object you get to do things like transport: https://stackoverflow.com/questions/3635131/paramikos-sshclient-with-sftp

import paramiko
def getSSHClient(proxy_ip, proxy_login, proxy_pw):
    """
    Instantiate, setup and return a straight forward proxy SSH client
    :param proxy_ip:
    :param proxy_login:
    :param proxy_pw:
    :return:
    """
    client = paramiko.SSHClient()
    client.load_system_host_keys()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.connect(proxy_ip, 22, username=proxy_login, password=proxy_pw)
    return client

def getProxySSHClient(proxy_ip, proxy_login, proxy_pw, destination_ip, destination_login, destination_pw):
    """
    Establish a SSH client through the proxy.
    :param proxy_ip:
    :param proxy_login:
    :param proxy_pw:
    :param destination_ip:
    :param destination_login:
    :param destination_pw:
    :return:
    """
    proxy = getSSHClient(proxy_ip, proxy_login, proxy_pw)
    transport = proxy.get_transport()
    dest_addr = (destination_ip, 22)
    local_addr = ('127.0.0.1', 10022)
    proxy_transport = transport.open_channel('direct-tcpip', dest_addr, local_addr)

    client = paramiko.SSHClient()
    client.load_system_host_keys()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.connect(destination_ip, 22, username=destination_login, password=destination_pw, sock=proxy_transport)
    return client

 

 
Posted in Uncategorized

Building MincToolkit for CentOS7 FUN

You probably want to build OpenBLAS.

Also make sure to sudo yum install hdf5, gsl, itk, netcdf, pcre, zlib, openblas-devl (still end up having to make my own),  etc…. that seems to have helped getting around the aforementioned empty string hash issue.

CCmake3 and Cmake3 also seems to have helped.

I recall having to install ccache as well.

 

Overall, it is MUCH easier to build it in Ubuntu, sigh.

 
Posted in Uncategorized

More Compilation Fun with d41d8cd98f00b204e9800998ecf8427e

So, the magical string of: d41d8cd98f00b204e9800998ecf8427e

Is the MD5 hash that CMake (or any other pogram) typically yield when the module is not found and ran an MD5 sum on an empty string apparently…

If there is anywhere you get a DOWNLOAD HASH mismatch and showing actual hash is: d41d8cd98f00b204e9800998ecf8427e, in English, the program is complaining that the resource is not found and a MD5 has check revealed that the hash of an empty string is different than whatever you are expecting… Now that makes a bit more sense… but no where close to solution…

 
Posted in Uncategorized

DrawEM MIRTK compilation fun

Fun, as summarized in Dwarf Fortress.

So. Been compiling DrawEM module of MIRTK on both CentOS and Ubuntu18. Fun indeed. I will write about compilation fun in general another day as its a source of much fun elsewhere too.

So, if you ever see build errors like:

[ 75%] Building CXX object Packages/DrawEM/src/CMakeFiles/LibDrawEM.dir/BiasCorrection.cc.o
In file included from /home/dyt811/MIRTK/Packages/DrawEM/src/BiasCorrection.cc:21:0:
/home/WindStalker/MIRTK/Packages/DrawEM/include/mirtk/BiasCorrection.h:29:34: fatal error: mirtk/Transformation.h: No such file or directory
#include "mirtk/Transformation.h"
^
compilation terminated.
Linking CXX executable ../../../lib/tools/calculate-gradients
Linking CXX executable ../../../lib/tools/measure-volume
Linking CXX executable ../../../lib/tools/change-label
make[2]: *** [Packages/DrawEM/src/CMakeFiles/LibDrawEM.dir/BiasCorrection.cc.o] Error 1
make[1]: *** [Packages/DrawEM/src/CMakeFiles/LibDrawEM.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 75%] Built target change-label
[ 75%] Built target calculate-gradients
[ 75%] Built target measure-volume
Linking CXX executable ../../../lib/tools/padding
Linking CXX executable ../../../lib/tools/calculate-filtering
[ 75%] Built target padding
[ 75%] Built target calculate-filtering
Linking CXX executable ../../lib/tools/aggregate-images
[ 75%] Built target aggregate-images
/home/WindStalker/MIRTK/Applications/src/average-images.cc: In function ‘int main(int, char**)’:
/home/WindStalker/MIRTK/Applications/src/average-images.cc:876:5: error: ‘imdof_name’ was not declared in this scope
imdof_name .insert(imdof_name.end(), imdofs .begin(), imdofs .end());
^
/home/WindStalker/MIRTK/Applications/src/average-images.cc:877:5: error: ‘imdof_invert’ was not declared in this scope
imdof_invert.insert(imdof_invert.end(), invdofs.begin(), invdofs.end());
^
make[2]: *** [Applications/src/CMakeFiles/average-images.dir/average-images.cc.o] Error 1
make[1]: *** [Applications/src/CMakeFiles/average-images.dir/all] Error 2
make: *** [all] Error 2

This is because DrawEM during compilation expect home directory to be named “mirtk” instead of “MIRTK” which is what git clone generated from main GitHub repo… so rename to “mirtk” and regenerate CMAKE cache will fix it… Yup. it is a lot less scarier than it looked….

How I found out? While compiling on both CentOS and Ubuntu ran into this same issue…

 
Posted in Uncategorized

SmartThings, Google Home and WeMO mingle words

So this is not the start of a joke, but a super annoying problem that took me too long to resolve.

I call Google Assistant, “Turn off the bedroom”, and google assistant says he has no clue what I am talking about… EVEN though I set bedroom up in both SmartThings, and Google Home.  A few trial and errors later, I realized, that you can either bypass that by saying

  1. “Turn off everything in the bedroom”
  2. “Turn off bedroomS”
  3. rename things away from “Bedroom light” “Bedroom power” etc.

The root cause is because Google is currently not able to tell when singular, “Bedroom” refer to the room, and not the part of the name of the devices name. So hence why it says cannot tell which device you want to modify.

Another thing I ran into is WeMo has its own name when Linked to Google Home. Then, WeMo is also linked to SmartThing which is linked to GoogleHome which cause device multiplication and confusion, and same issue above. To resolve this, what I did was 1) make sure WeMO use some bizarre name like the A0F from the manufacturer’s unique ID. This ensure the device is referred by different names in SmartThing and Wemo and Google Home use the name I assigned in SmartThing despite its direct loading Wemo nonsensical name that no one will call it.

In the end I removed all rooms from SmartThings and only used Google Home to define rooms. Most automation things are handled by SmartThing routines but still, many bugs and I barely have time to figure out. Also, Google keep mishearing “Turn on” with “Turn off” vice versa. Very very annoying.  Oh and also SmartThing on iPhone phone presence never seemed to work for my wife, at, al. Damn it.

Overall, I would say home automation is still quirky as fffff. I do like the fact that thanks to Wemo’s mini plugs and the sort, I do not have to 1) hire an electrician, 2) connect a neutral from the nearest wall sockets 3) drill holes in the ceiling and connect to the switch etc.  JUST so that I can install GE ZWave compatible plugs. That … was ffff annoying. Still need electricians for 2 way switches though.

I shutter at the thought or replacing a lock with smart locks.

 
Posted in Uncategorized

Swarm, Driving, Traffic Signals and Crowd Sourced Data Processing

Funny thing I noticed recently.

I stopped paying as much attention to the surrounding when I drive as when I first started.

Not sure if you have noticed similar trend. When first start learning to drive, newbie drivers like me tend to be super stressed out because of the CRAZY amount of things I need to pay attention to: blind spots, car on the left, car on the right, upcoming intersections, is left turn forbidden here, what is that elderly pediatrician thinking planning to cross on red? etc etc etc…. Many many things. Experienced drivers check much fewer things: left turn: blank spot, biker, oncoming traffic, done. Outta here. In particular, I find interesting is the scenarios when it comes to  red light. Most people probably have the unfortunate experience of accidentally running the redlight a few times in their lives. From the few incidences I have seen, it is extremely rare for people to run red light when there are cars stopped at the intersection especially in the opposite direction or same direction, even if those cars are not in the same lane. On the other hand, people run red light usually when there is no car stopped at the red light lane. This observation got me thinking, maybe, it is not that we are paying attention to the light but more to the cars. In another way to look at this, perhaps we are not really following the signs/rules as diligently as we should have when we were beginners but instead, relying on other drivers’ reaction to the surrounding to gauge how we should behave.

Another example, beginner driver like me tend to stress over the speed I am driving at… constantly struggling between not exceeding speed limits too much but also at the same time ensure keep about similar pace as traffic surrounding them. That is especially stressful during late night driving when certain roads and some crowd tend to drive far far exceeding the speed limit but as a group. Eg. STM mignight buses.

Speaking from personal experience I think driving has become a balanced experience where while in the stream, I tend to follow how folks around me are driving and paying attention to very specific contextual details that are specific to me but relying on the other motorists to notice issues such as, oh they slowed down, I probably should slow down too, even if I am not in the same line. Herd instinct perhaps? While in the areas specific to my destination such as local streets etc, I have to be much more diligent.

Maybe the car of the future will be far more aware of each other’s presence and require relatively little onboard processing power but rely on the swarm of them to process the large amount of information required on the overall traversal goals. It would certainly be an interesting time.

 
Posted in Uncategorized

PyCharm, Terminal and Environmental Variables

Banged my head on this way too long before I realize: when you add new path to environmental variables, you probably want to restart pycharm to ensure the terminal is reloading the new path info and can now resolve whatever new resources being added to the path. =_=+ embarrassingly long.

 
Posted in Uncategorized

MANTIS bug

So yeah, another thing I noticed is that MANTIS’ skull stripping algorithm actually cannot have space in any of the path or else… it will crash. So.Typical.Windows.Bug….

 
Posted in Uncategorized

Matlab on CentOS with X11

Fun fun fun installing Matlab on CentOS7…. on a server…. without display.

Our resident magical IT guru managed to get XServe up and running but even then… we saw a splash screen and installation FINISHED like what is described here:https://www.mathworks.com/matlabcentral/answers/229857-why-do-i-see-preparing-installation-files-installing-finished-in-the-terminal-window-wh

This problem blocked us for a while and never got resolved. Then our magical IT guru Rolando took a stab at it and solved it, all thanks to this post by John Miezitis (posted recently on 24th last month) which somehow even more magically found libXtst library is MISSING which is “silently” crashing the installation, as in this post: https://www.mathworks.com/matlabcentral/answers/229857-why-do-i-see-preparing-installation-files-installing-finished-in-the-terminal-window-wh#answer_330037

Kudos to John, whever you are. We owe you a beer in Montreal.

Yeah, we need GUI on a server (sigh) because SPM/Matlab and a few other programs are very heavy GUI driven and to facilitate debug time, a GUI really helped formation of the analyses script integration. I really wish this abomination isn’t born but so many request for SPM based analyses pipelines (looking at MANTIS right now…)

 

 

 
Posted in Uncategorized

The journey to get iBeat from iDEA working.

Wow… fun time working with matlab/linux/precompiled binaries…

So, got this awesome software that I have never heard of called iBeat that does pretty epic neonatal segmentation. Except no one I know ever used it. So. I am the dude who shall eat and relish this proverbial crab.

Binary pre-compiled. Joyous.

Then, after editing .bashrc to source their bash source script,

MCR crashed. Matlab Compile? Runtime is what this GUI and others were built on. FUNNNNN…  Cryptic message just like what is reported in the solution thread:

Fatal error loading library /home/lz225/v76/bin/glnxa64/libmwmclmcr.so Error: libXp.so.6: cannot open shared object file: No such file or directory 

Down off the search rabbit hole I went.

Glad to see I was not the only one running into this issue: https://www.linuxquestions.org/questions/linux-newbie-8/error-libxp-so-6-cannot-open-shared-object-file-no-such-file-or-directory-924315/

TLDR so far, this libmwmclmcr.so file (god… what could this abbreviation possbily mean???). Reqeuired a dynamicly linked library called libXp.so.6 which is somehow used for printing?

Then further stackxxxxxgoogle change later on: Found this thread: https://askubuntu.com/questions/944838/libxp-so-6-missing-for-ubuntu-17-04

Which led me to try that too. Downloaded from https://packages.ubuntu.com/trusty/libxp6

and ran

sudo dpkg -i ./libxp6_1.0.2-1ubuntu1_amd64.deb
sudo apt-get install -f

and even though prompt said NOTHING is updated. It worked miraculously by some dark menacing black magic I guess.

and now I can actually launch iBeat now.

How did people use to solve these problems PRE search engine??? Must take forever on mailing list or snailmail author for giggles?

Update 1: Okay… maybe I spoke too soon. The moment iBeat ran once, system crashed. Super impressive as this is the SECOND time this happened. Previously with a superstable CentOS. Now with Ubuntu. Something feels odd. Don’t tried the crossed out fix until you got a VM snapshot or the like up. This is getting personal 😛 I reinstalled Ubuntu, snappedshotted this to hell and going to do SCIECNE on ibeat so hard until it works.

Update 2: Retried that patch method… seeing if Ubuntu terminal dies.

Update 3: Nope. Ubuntu is still alive and kicking THIS time. Hmmm… then this weird death of both a VM and my old laptop must just be a freaky coincidence then. I did install Anaconda last time prior to this… wondering if they are related… But either way, VM snapshots all the time now.

 

 
Posted in Uncategorized