Connecting to UC SMB shares on linux

Contents

Introduction

Connecting to the UC shared file system can be painful. Using the P drive as your home under linux requires a lot of machinery to be in place to make sure the person accessing the share is you. RedHat SOE from the university accomplish this by using a framework called free-ipa which requires that the machine is pre-enrolled before being able to access the shares. Over time, the campus departmental staff cobbled various solutions together, involving Active Directory and Kerberos to achieve similar results. These solutions don't always work as desired as there is no clear documented procedure to achieve such result in this context (apart from free-ipa). In the future a systemd sub-system called homed may provide an interesting solution but it is not widely deployed yet and there is no documentation for a setup like the one we have at UC.

While using the P drive as your home is complicated, it shouldn't be difficult to access your files given proper identification. The basic samba client, smbclient, lets you see the shares you have access to properly. You can also use the sftp gateway described in the data transfer page of this wiki. However, we should be able to mount the share in the same way we can for service like dropbox, as a folder of your file system under your home or a predetermined location. On systems with a graphical interface (kde or gnome) the file manager can expose those shares and to some extent use them as you would a local file system. In the next sections we will show how to achieve this from the command line by exploiting some of the sub-systems  used by those file managers.

The procedures detailed below can be done from a machine on the UC campus, connected through VPN to the UC campus or a RCC machine deployed on the "internal network" (132.181.102). RCC machines on the "private network" (10.195.0) may be able to connect with some extra configurations for the NAT associated to it and replacing URLs by their IP equivalent - it is untested at the time of writing. However we can use the same tools to mount the sftp  gateway, more on below. For security reason, I don't advise to try this on machines directly connected to the internet.

Using gio

Requirements

On Ubuntu 18.04 and 20.04 we will need the following packages

sudo apt install dbus-x11 gvfs-fuse gvfs-backends gvfs-bin cifs-utils

On Ubuntu 22.04 the gvfs-bin has been removed

sudo apt install dbus-x11 gvfs-fuse gvfs-backends cifs-utils

On RedHat on other distributions, you will need an equivalent list. Apart from cifs you will need to at least have the following executables dbus-launch, gio and possibly gvfs-mount. The last one being deprecated in favor of gio. 

Dbus session

When you start a graphical session, a special communication daemon called dbus is launched that will be used by many of your graphical applications to exchange information. There is usually one such daemon running per graphical session. Different sessions and users will each have their dbus session and usually there won't be any sharing between any of them. The tool that we want to use  (gvfsd the "gnome virtual file system daemon"), relies on it .  To be able to use gvfsd, we need a dbus session, however while graphical session automatically start a dbus session, ssh and console sessions don't. So, we will need to manually start one

dbus-launch

The above command starts a dbus session in the background and will spit out some data identifying the dbus session, you should take note of it for now. A sample output is given below, but each session will have its unique ID string

$ dbus-launch 
DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-ZNE11m9SMT,guid=8bed66bcf20fb6241726a9895e9cdab1
DBUS_SESSION_BUS_PID=5987

The variable DBUS_SESSION_BUS_ADDRESS is particularly important and is what you need to share a dbus session between different ssh or console session. The dbus session process shows up as dbus-daemon when listing processes with ps ux (this list all the processes of the user currently logged)

rccadmin   5987  0.0  0.1  62044  2760 ?        Ss   11:11   0:00 /usr/bin/dbus-daemon --syslog --fork --print-pid 4 --print-address 6 --session

This daemon process will stay alive until you reboot the machine or you manually kill it. It will persist if you logout, however when you login again, if you want to use it you need to reconnect to it by setting DBUS_SESSION_BUS_ADDRESS to the right value again.

export DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-ZNE11m9SMT,guid=8bed66bcf20fb6241726a9895e9cdab1

$ export $(dbus-launch)

First gio command

Now that we have a dbus session running and connected we can mount our first SMB share, for examples, the "departments" share ( K drives )

gio mount smb://file.canterbury.ac.nz/departments

You will be prompted for login information

$ gio mount smb://file.canterbury.ac.nz/departments
Password required for share departments on file.canterbury.ac.nz
User [rccadmin]: 
Domain [WORKGROUP]: uocnt
Password: 

For User you should use your UC user code (something of the form frb15 - three letters from you names and two to three digits). The Domain field should be set to uocnt as above and Password is your usual UC password associated with your UC user code.

The command ps ux will now show a number of process associated with gvfsd. Like the dbus daemon, those processes will persist until killed or a reboot.

rccadmin   6191  0.0  0.3 283560  6856 ?        Ssl  11:32   0:00 /usr/lib/gvfs/gvfsd
rccadmin   6196  0.0  0.2 350580  5276 ?        Sl   11:32   0:00 /usr/lib/gvfs/gvfsd-fuse /run/user/1001/gvfs -f -o big_writes
rccadmin   6197  0.0  0.4 297916  8372 ?        Ssl  11:32   0:00 /usr/lib/gvfs/gvfs-udisks2-volume-monitor
rccadmin   6208  0.0  0.3 370424  7620 ?        Ssl  11:32   0:00 /usr/lib/gvfs/gvfs-afc-volume-monitor
rccadmin   6213  0.0  0.2 280228  5728 ?        Ssl  11:32   0:00 /usr/lib/gvfs/gvfs-gphoto2-volume-monitor
rccadmin   6217  0.0  0.2 265648  5828 ?        Ssl  11:32   0:00 /usr/lib/gvfs/gvfs-goa-volume-monitor
rccadmin   6221  0.0  0.2 267440  5112 ?        Ssl  11:32   0:00 /usr/lib/gvfs/gvfs-mtp-volume-monitor
rccadmin   6236  0.0  0.9 689532 18696 ?        Sl   11:43   0:00 /usr/lib/gvfs/gvfsd-smb --spawner :1.2 /org/gtk/gvfs/exec_spaw/0

The share is now mounted and an another gio invocation will show that it is

$ gio mount -l
Drive(0): VMware Virtual disk
  Type: GProxyDrive (GProxyVolumeMonitorUDisks2)
Drive(1): VMware Virtual SATA CDRW Drive
  Type: GProxyDrive (GProxyVolumeMonitorUDisks2)
Mount(0): departments on file.canterbury.ac.nz -> smb://file.canterbury.ac.nz/departments/
  Type: GDaemonMount

As you can see, it is listed as Mount(0). But where is it mounted? This is the most inconvenient point, it is mounted in a special runtime folder. Fortunately, we don't need to remember or know the full name of the folder in question as it is contained in an environment variable that is set when you login, XDG_RUNTIME_DIR. The mount point is actually in the gvfs subfolder defined by the previous variable

$ ls -la $XDG_RUNTIME_DIR/gvfs/
total 0
dr-x------ 3 rccadmin rccadmin   0 Apr 20 11:32  .
drwx------ 8 rccadmin rccadmin 200 Apr 20 11:43  ..
drwx------ 1 rccadmin rccadmin   0 Jan 21 12:09 'smb-share:server=file.canterbury.ac.nz,share=departments'
$ ls $XDG_RUNTIME_DIR/gvfs/smb-share\:server\=file.canterbury.ac.nz\,share\=departments/
AccountancyInformationSystems  CollegeEducation			 FinancialServices		     MaoriIndigenousStudies		     RochesterRutherford
AVCMaori		               CollegeEngineering		 FineArts			         MasterEngineeringManagement	 SchoolOfProductDesign
BiologicalSciences	           CollegeScience			 Forestry			         MathematicsStatistics		     SocialPoliticalSciences
BiomolecularInteractionCentre  ComputerScience			 GeospatialResearchInstitute MechanicalEngineering		     SpatialEngineeringResearchCentre
CampusLiving		           ConfuciusInstitute		 HITLab				         Music				             StudentHealthCounselling
CentreEvaluationMonitoring     DeputyViceChancellor		 Humanities			         NgaiTahuResearchCentre		     StudentServicesCommunications
CentreResearchEurope	       EarlyChildhoodEducation   HumanResources			     NZInstituteLanguageBrainBehaviour	     TheatreFilmStudies
ChemicalProcessEngineering     EarthEnvironment			 InstituteRiskResilienceRenewal      PhysicalChemicalSciences		     UCHighPerformanceComputing
Chemistry		               EconomicsFinance			 Law				         PhysicsAstronomy			UniversityCanterburyStudentAssociation
ChristchurchCollegeOfEnglish   EducationPlus			 LearningResources		     PsychologySpeechHearing		  ViceChancellorsOffice
CivilEngineering	           ElectricalComputerEngineering	 MacDiarmidInstitute QuakeCore				          WirelessResearchCentre
CollegeArts		               ElectricalPowerEngineeringCentre  MacmillanBrownCentrePacificStudies  ResearchInnovation
CollegeBusinessLaw	           ExecutiveDevelopment		 Management			         ResearchPartnershipsBuildingInnovation

It is still a fairly inconvenient address, but this is nothing creating a link cannot fix.

The P drive

The P drive, the UC official home directory for users can also be accessed with a bit of caution. Because there are so many users at UC, the home directories have been split across the 26 letters of the alphabet. From usersa$ to usersz$ where the letter is the first letter of your UC usercode. So, someone with the user code nop128 will need to mount usersn$ to access their home. Note that the $ at the end is not a mistake, it is a bit of SMB magic and forgetting it will result in an error message

$ gio mount smb://file.canterbury.ac.nz/usersf
Password required for share usersf on file.canterbury.ac.nz
User [rccadmin]: f#####
Domain [WORKGROUP]: uocnt
Password: 
gio: smb://file.canterbury.ac.nz/usersf/: Failed to mount Windows share: No such file or directory

The correct way is as follow (note the "escaping" $ with "\")

$ gio mount smb://file.canterbury.ac.nz/usersf\$
Password required for share usersf$ on file.canterbury.ac.nz
User [rccadmin]: f#####
Domain [WORKGROUP]: uocnt
Password: 
rccadmin@UCRCC0390:~$ ll $XDG_RUNTIME_DIR/gvfs/
total 0
dr-x------ 4 rccadmin rccadmin   0 Apr 20 11:32  .
drwx------ 8 rccadmin rccadmin 200 Apr 20 11:43  ..
drwx------ 1 rccadmin rccadmin   0 Jan 21 12:09 'smb-share:server=file.canterbury.ac.nz,share=departments'
drwx------ 1 rccadmin rccadmin   0 Mar 25 12:07 'smb-share:server=file.canterbury.ac.nz,share=usersf$'

Your P drive home directory is in a subfolder named after your user code under that mount point.

Getting the value of DBUS_SESSION_BUS_ADDRESS

As we have seen previously, the dbus session is persistent and you can reconnect to it across various login. However, you need the value of DBUS_SESSION_BUS_ADDRESS to be able to connect to it. You could store it after the start of the session or you can recover it from the environment of a process launched under dbus like gvfsd. This is rather obscure but you can recover and extract the environment from a running process by querying the right file handle under /proc. It is more precisely under /proc/${PID_of_gvfsd}/environ. It can be easily recovered with a script. In summary if you launch a dbus session and start a gvfsd process under it, you can always recover the dbus session in question and the re-attach to the gvfsd process with appropriate scripting.

Overriding the $XDG_RUNTIME_DIR/gvfs mount point

It is actually possible to make the mount point a bit more friendly and locate it somewhere inside your home directory for example. The gvfs component mounting SMB share is called gvfsd-fuse and is automatically started by gvfsd on your first call of gio. According to the man page of gvfsd-fuse, you can start it with a given path where your mount points will be located.

SYNOPSIS
       gvfsd-fuse PATH


DESCRIPTION
       gvfsd-fuse maintains a fuse mount to make gvfs backends available to POSIX applications. The mount point for the fuse filesystem is provided by the [PATH] argument.


       gvfsd-fuse is normally started by gvfsd(1). In this case, the mount point is $XDG_RUNTIME_DIR/gvfs or $HOME/.gvfs.

$HOME/.gvfs is only used when $XDG_RUNTIME_DIR/gvfs cannot be created or is inaccessible. But we can override this setting by starting gvfsd-fuse with a path of our choosing and then then letting gvfsd be started by gio. We even have an extra option in gvfsd to tell it not start gvfsd-fuse so has not to have two instances running at the same time (in that case, it appears the first instance of gvfsd-fuse is used, so it is technically harmless).  From the man page of gvfsd

ENVIRONMENT
       GVFS_DISABLE_FUSE
           If this environment variable is set, gvfsd will not start the fuse filesystem.

So, in order we could start dbus, start gvfsd-fuse with a path of our choosing, set GVFS_DISABLE_FUSE and then call our first gio command. And everything will be mounted in the folder of our choosing. Because it is a low level command that you are supposed to call directly, gvfsd-fuse is not in the PATH. You have to know its location to call it

# ubuntu 18.04/20.04
/usr/lib/gvfs/gvfsd-fuse
# ubuntu 22.04
/usr/libexec/gvfsd-fuse

Automating the mount

With all the elements above we can write a few scripts which can be executed on login which will make sure the shares you want are mounted. If you don't want it to be automated on login you can keep them around to be launched manually instead.

Storing your UC login credentials

Part of the automation and scripting requires your UC credentials (user code and password) to be stored in a file. This is unfortunate but we can make it as secure as we can nonetheless. You can change the locations and name of the file, but subsequent scripts will rely on this location and will need to be changed accordingly if you choose a different location. First lets create the .config directory if it doesn't exist already with 

mkdir -p ~/.config

Now lets create .config/credentials with nano and fill it with values following the template below

UC usercode
uocnt
UC password

finally make sure you are the only one (apart from another administrator) that can read the file

chmod 0600 ~/.config/credentials

Creating the folder containing the mount points

We should create a folder for our mount points

mkdir ~/UCDrives

While creating it is a good idea, the script will make sure it exists.

Automation script for mounting shares

We will rely on several elements seen in the previous sections plus some magic invocation to retrieve the details of already launched dbus session from a running gvfsd process. If gvfsd is not running dbus is launched and the share mounted. If gvfsd is running, the dbus session is re-attached so that the mount points can be manipulated with gio or new one be added as desired. Save the following script as ~/.UCsmb.rc

UCSMB="file.canterbury.ac.nz"
export GVFS_DISABLE_FUSE=1
MNTPOINT="${HOME}/UCDrives"
# Making sure the mountpoint exists
mkdir -p "${MNTPOINT}"
# getting the PID of gvfsd - if there are several users on the system, we may get several values.
PID=$(pidof gvfsd)
# figuring out the first character of your usercode
initial=$(head -c 1 ~/.config/credentials)
# figure where gvfsd-fuse is
if [[ -f /usr/lib/gvfs/gvfsd-fuse ]]; then
    # ubuntu 18.04/20.04
    GVFSDFUSE=/usr/lib/gvfs/gvfsd-fuse
else
    # ubuntu 22.04
    GVFSDFUSE=/usr/libexec/gvfsd-fuse
end
if [[ "${PID}" == "" ]]; then
    # no dbus available launch a new one
    export $(dbus-launch)
    # start the gvfsd-fuse daemon with appropriate mount point and options.
    # The options are the one that would be used if started by gvfsd.
    ${GVFSDFUSE} "${MNTPOINT}" -f -o big_writes &
    # mount essential shares
    for i in users${initial}\$ departments research bulk scratch shared; do
        gio mount smb://${UCSMB}/"${i}" < ~/.config/credentials
    done
else
    # checking that the PID or one of the PIDs is owned by the current user.
    dbuslaunchedpid=0
    for i in ${PID} ; do
        owner=$(ps -o user= -p ${i})
        if [[ "${owner}" == "${LOGNAME}" ]]; then
            dbuslaunchedpid=${i}
        fi
    done
    if [[ "${dbuslaunchedpid}" == "0" ]]; then
        # In that case gvfsd belongs to someone else, launch our own dbus.
        export $(dbus-launch)
        # start the gvfsd-fuse daemon with appropriate mount point and options.
        # The options are the one that would be used if started by gvfsd.
        ${GVFSDFUSE} "${MNTPOINT}" -f -o big_writes &
        # mount essential shares
        for i in users${initial}\$ departments research bulk scratch shared; do
            gio mount smb://${UCSMB}/"${i}" < ~/.config/credentials
        done
    else
        # reusing existing dbus session
        export DBUS_SESSION_BUS_ADDRESS=$(tr '\0' '\n' < /proc/${dbuslaunchedpid}/environ | grep "DBUS_SESSION_BUS_ADDRESS" | cut -d "=" -f 2-)
        echo $DBUS_SESSION_BUS_ADDRESS
    fi
fi

The script can be executed manually with

. ~/.UCsmb.rc

or automatically at login by inserting the following lines at the end of ~/.bashrc

if [ -f ~/.UCsmb.rc ]; then
    . ~/.UCsmb.rc
fi

This can be complimented by creating easy to follow links in your home directory like

ln -s ~/UCDrives/smb-share\:server\=file.canterbury.ac.nz\,share\=users${initial}\$/${usercode} ~/Pdrive

where you replace ${initial} and ${usercode} by the appropriate values in your case. For a user called frb15 this would be f and frb15 for example.

Automating the mount (sftp version)

With all the elements above we can write a few scripts which can be executed on login which will mount the sftp gateway like a drive. Similar to how it works for smb. As with the smb protocol, if you don't want it to be automated on login you can keep them around to be launched manually instead.

Storing your UC login credentials

Part of the automation and scripting requires your UC credentials (user code and password) to be stored in a file. This is unfortunate but we can make it as secure as we can nonetheless. You can change the locations and name of the file, but subsequent scripts will rely on this location and will need to be changed accordingly if you choose a different location. First lets create the .config directory if it doesn't exist already with 

mkdir -p ~/.config

Now lets create .config/credentials-sftp with nano and fill it with values following the template below

UC usercode
UC password

finally this command to make sure you are the only one (apart from another administrator) that can read the file

chmod 0600 ~/.config/credentials-sftp

Creating the folder containing the mount points

We should create a folder for our mount points

mkdir ~/UCDrives

While creating it is a good idea, the script will make sure it exists.

Automation script for mounting shares

We will rely on several elements seen in the previous sections plus some magic invocation to retrieve the details of already launched dbus session from a running gvfsd process. If gvfsd is not running dbus is launched and the share mounted. If gvfsd is running, the dbus session is re-attached so that the mount points can be manipulated with gio or new one be added as desired. Save the following script as ~/.UCsftp.rc

UCSFTP="filegateway.canterbury.ac.nz"
export GVFS_DISABLE_FUSE=1
MNTPOINT="${HOME}/UCDrives"
# Making sure the mountpoint exists
mkdir -p "${MNTPOINT}"
# getting the PID of gvfsd - if there are several users on the system, we may get several values.
PID=$(pidof gvfsd)
if [[ "${PID}" == "" ]]; then
    # no dbus available launch a new one
    export $(dbus-launch)
    # start the gvfsd-fuse daemon with appropriate mount point and options.
    # The options are the one that would be used if started by gvfsd.
    /usr/lib/gvfs/gvfsd-fuse "${MNTPOINT}" -f -o big_writes &
    # mount file gateway as a drive
    gio mount sftp://${UCSFTP} < ~/.config/credentials-sftp
else
    # checking that the PID or one of the PIDs is owned by the current user.
    dbuslaunchedpid=0
    for i in ${PID} ; do
        owner=$(ps -o user= -p ${i})
        if [[ "${owner}" == "${LOGNAME}" ]]; then
            dbuslaunchedpid=${i}
        fi
    done
    if [[ "${dbuslaunchedpid}" == "0" ]]; then
        # In that case gvfsd belongs to someone else, launch our own dbus.
        export $(dbus-launch)
        # start the gvfsd-fuse daemon with appropriate mount point and options.
        # The options are the one that would be used if started by gvfsd.
        /usr/lib/gvfs/gvfsd-fuse "${MNTPOINT}" -f -o big_writes &
        # mount file gateway as a drive
        gio mount sftp://${UCSFTP} < ~/.config/credentials-sftp
    else
        # reusing existing dbus session
        export DBUS_SESSION_BUS_ADDRESS=$(tr '\0' '\n' < /proc/${dbuslaunchedpid}/environ | grep "DBUS_SESSION_BUS_ADDRESS" | cut -d "=" -f 2-)
                # checking if fuse is running with the right path
        fuserunning=$(ps ux | grep gvfsd-fuse | grep ${MNTPOINT})
        if [[ "${fuserunning}" == "" ]]; then
            # restart the fuse and the mount
            /usr/lib/gvfs/gvfsd-fuse "${MNTPOINT}" -f -o big_writes &
            gio mount sftp://${UCSFTP} < ~/.config/credentials-sftp
        else
           # gvfsd-fuse is running with the right settings
           # is sftp mounted?
           sftpmount=$(gio mount -l | grep $UCSFTP)
           if [[ "${sftpmount}" == "" ]]; then
               # remount the sftp drive
               gio mount sftp://${UCSFTP} < ~/.config/credentials-sftp
           fi
           # do nothing we are fine.
        fi
    fi
fi

The script can be executed manually with

. ~/.UCsftp.rc

or automatically at login by inserting the following lines at the end of ~/.bashrc

if [ -f ~/.UCsftp.rc ]; then
    . ~/.UCsftp.rc
fi

Unlike, samba, sftp will disconnect after some inactivity, this is built in the protocol, in that case re-running .UCsftp.rc  won't be useful as it doesn't expect the connection to die. One way around it, is to tell ssh to keep the connection alive. This can be done by adding a couple of lines to the file ~/.ssh/config  or creating it if it doesn't exist. You should add the following two lines (at the end) to the file

Host *
ServerAliveInterval 60

Even with those two lines, you may eventually get disconnected. In that case the .UCsftp.rc  script has been made rerunnable and should detect if things are disconnected and whther any components needs restarting.