Reload packages

Hello all,

I am building a NSO (network services orchestrator) automation and would like to ask the best approach regarding reload packages using github as SCM.

Currently every time that I run the pipeline it will clone all packages for a tmp directory, destroy the entire NSO package folder, create a new folder and copy all packages back from the tmp folder inside NSO package folder.

What’s the best way to clone only the packages that was changed or don’t clone any package at all in case there is no changes to minimise the clone time because in the future I will have more packages and it will take a long time.

Structure of packages folder inside NSO Server:

<font face="andale mono,times">~/Box Sync/GitBroadsoft/inf-nso-services/packages   master  ll
total 0
drwxr-xr-x 13 rengonca staff 416B 12 Jun 10:47 alu-sr
drwxr-xr-x 10 rengonca staff 320B 5 Jun 18:09 cisco-acl
drwxr-xr-x 8 rengonca staff 256B 5 Jun 18:09 cisco-initialize-acl
drwxr-xr-x 7 rengonca staff 224B 5 Jun 18:09 cisco-initialize-authentication
drwxr-xr-x 13 rengonca staff 416B 5 Jun 18:09 cisco-ios
drwxr-xr-x 12 rengonca staff 384B 5 Jun 18:09 cisco-iosxr
drwxr-xr-x 9 rengonca staff 288B 5 Jun 18:09 cisco-vrf-bgp
drwxr-xr-x 13 rengonca staff 416B 12 Jun 10:47 f5-bigip
drwxr-xr-x 10 rengonca staff 320B 5 Jun 18:09 juniper-acl
drwxr-xr-x 7 rengonca staff 224B 5 Jun 18:09 juniper-initialize-acl
drwxr-xr-x 10 rengonca staff 320B 5 Jun 18:09 juniper-junos
drwxr-xr-x 7 rengonca staff 224B 12 Jun 10:47 snmpT1
 ~/Box Sync/GitBroadsoft/inf-nso-services/packages   master </font>

Pipeline:

<font face="andale mono,times">pipeline {
     agent {
         node { label 'nso01' }
     }
     stages {
         stage('Start maintenance') {
             steps {
                sh 'echo "Starting..."'
             }
         }
         stage('NSO - Git Clone') {
             steps {
                 checkout scm
             }
         }
         stage('NSO - delete current packages') {
            steps {
               sh 'rm -rf ~/ncs-run/packages/* ; mkdir -p ~/ncs-run/packages/'
             }
         }
         stage('NSO - Move packages') {
             steps {
                sh 'cp -a ./packages/* ~/ncs-run/packages/'
             }
         }
         stage('NSO - Stop') {
              steps {
                   catchError {
                        sh 'source ~/nso-4.7/ncsrc; cd ~/ncs-run; ncs --stop'
                  }
              }
         }
         stage('NSO - Start and Reload packages') {
              steps {
                    withEnv (['JENKINS_NODE_COOKIE=do_not_kill']) {
                        sh 'source ~/nso-4.7/ncsrc; cd ~/ncs-run; ncs --with-package-reload-force'
                     }
              }
         }</font>

 Regards

RG

I’m not sure I understand the question, but it sounds like you’re trying to minimize network traffic while also minimizing local storage use between invocations. If I’m correct in my understanding, then like many things in computers, this is a tradeoff much like “memory space versus speed” in many algorithms.

To be specific:

  • To have the absolute minimim network traffic, then cache the packages locally between invocations and issue a git pull to get only the changes that occurred between the last invocation and this one (but this will require local storage of the cache between invocations)
  • To have the absolute minimum local storage size, don’t cache anything and clone each package fresh every invocation (but this will require you to transfer the entire package every time, not just the changes)

To get a balance between the two extremes, determine a caching policy that decides when to throw out a locally cached package.

I hope that helps!