Fix Ubuntu/Debian package installation#13207
Conversation
|
@blueorangutan package |
|
@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
There was a problem hiding this comment.
Pull request overview
This PR updates the Debian/Ubuntu cloudstack-common post-install script to use Python 3 byte-compilation tooling during package installation, aiming to resolve install-time failures related to Python compilation steps.
Changes:
- Replace
pycompileinvocations withpy3compileindebian/cloudstack-common.postinst. - Keep post-install Python module copying and explicit
python3 -m py_compile/compileallsteps intact.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #13207 +/- ##
============================================
+ Coverage 18.09% 19.50% +1.40%
- Complexity 16732 19444 +2712
============================================
Files 6037 6303 +266
Lines 542780 569288 +26508
Branches 66464 69792 +3328
============================================
+ Hits 98233 111040 +12807
- Misses 433499 446094 +12595
- Partials 11048 12154 +1106
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17954 |
|
@blueorangutan test debian12 kvm-debian12 |
|
@weizhouapache a [SL] Trillian-Jenkins test job (debian12 mgmt + kvm-debian12) has been kicked to run smoke tests |
|
[SF] Trillian Build Failed (tid-16140) |
boring-cyborg
Bot
added
component:networking
component:storage
component:XenServer
Python
|
@blueorangutan package |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 20 changed files in this pull request and generated 11 comments.
Comments suppressed due to low confidence (1)
scripts/storage/secondary/cloud-install-sys-tmplt.py:176
file(...)is not defined in Python 3, so these calls will raiseNameErrorat runtime. Replace withopen(..., 'wb')(and consider using context managers) for decompression output andtemplate.propertieswrites.
def uncompressFile(self, fileName):
print('Uncompressing the file %s... which could take a long time, please wait' % self.systemvmtemplatepath)
if self.fileextension == 'gz':
compressedFile = gzip.GzipFile(self.systemvmtemplatepath, 'rb')
decompressedData = compressedFile.read()
compressedFile.close()
decompressedFile = file(fileName, 'wb')
decompressedFile.write(decompressedData)
decompressedFile.close()
| if which py3compile >/dev/null 2>&1; then | ||
| py3compile -p cloudstack-common /usr/share/cloudstack-common | ||
| fi | ||
|
|
| if self.args.templatesuffix: | ||
| self.templatesuffix = self.args.templatesuffix | ||
| print 'Password for DB: %s'%self.databaseuserpassword | ||
| print('Password for DB: %s'%self.databaseuserpassword) |
| def runCmd(self, cmds): | ||
| process = subprocess.Popen(' '.join(cmds), shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) | ||
| stdout, stderr = process.communicate() | ||
| print(stdout) | ||
| if process.returncode != 0: | ||
| raise Exception(stderr) | ||
| return stdout | ||
|
|
||
| def runMysql(self, query): | ||
| try: | ||
| print 'Running Query: %s' % query | ||
| print('Running Query: %s' % query) | ||
| mysqlCmds = ['mysql', '--user=%s'%self.databaseusername, '--host=%s'%self.databasehostname, '--password=%s'%self.databaseuserpassword, '--skip-column-names', '-U', 'cloud', '-e "%s"'%query] | ||
| templateId = self.runCmd(mysqlCmds) | ||
| print 'TemplateId is : %s' % templateId | ||
| except Exception, e: | ||
| print('TemplateId is : %s' % templateId) | ||
| except Exception as e: | ||
| err = '''Encountering an error when executing mysql script\n%s''' % str(e) | ||
| self.errorAndExit(err) | ||
| return templateId |
| try: | ||
| print 'Running Query: %s' % query | ||
| print('Running Query: %s' % query) | ||
| mysqlCmds = ['mysql', '--user=%s'%self.databaseusername, '--host=%s'%self.databasehostname, '--password=%s'%self.databaseuserpassword, '--skip-column-names', '-U', 'cloud', '-e "%s"'%query] | ||
| templateId = self.runCmd(mysqlCmds) | ||
| print 'TemplateId is : %s' % templateId | ||
| except Exception, e: | ||
| print('TemplateId is : %s' % templateId) |
| modfile = "%s/%s.py" % (modpath, cs.getName().lower()) | ||
| me = os.path.abspath(__file__) | ||
| if os.path.isfile(modfile): | ||
| if hashlib.md5(open(me).read()).hexdigest() != hashlib.md5(open(modfile).read()).hexdigest(): |
| sys.exit(2) | ||
| params = {} | ||
| pos = 0; | ||
| for i in range(len(paramList) / 2): |
| if val in ["10", "100", "1000"]: | ||
| settings += ['duplex', 'val'] |
| network_conf.close() | ||
| except Exception, e: | ||
| except Exception as e: | ||
| raise Error("failed to determine network backend:" + e) |
| if len(sys.argv) < 4: | ||
| print "Usage: prepare_kickstart_kerneal_initrd.py path_to_kernel path_to_initrd path_kernel_initrd_copy_to" | ||
| sys.exit(1) | ||
| print("Usage: prepare_kickstart_kerneal_initrd.py path_to_kernel path_to_initrd path_kernel_initrd_copy_to") |
| if __name__ == "__main__": | ||
| if len(sys.argv) < 12: | ||
| print "Usage: prepare_tftp_bootfile.py tftp_dir mac cifs_server share directory image_to_restor cifs_username cifs_password ip netmask gateway" | ||
| print("Usage: prepare_tftp_bootfile.py tftp_dir mac cifs_server share directory image_to_restor cifs_username cifs_password ip netmask gateway") |
| if len(sys.argv) < 4: | ||
| print "Usage: prepare_kickstart_kerneal_initrd.py path_to_kernel path_to_initrd path_kernel_initrd_copy_to" | ||
| sys.exit(1) | ||
| print("Usage: prepare_kickstart_kerneal_initrd.py path_to_kernel path_to_initrd path_kernel_initrd_copy_to") |
| if __name__ == "__main__": | ||
| if len(sys.argv) < 12: | ||
| print "Usage: prepare_tftp_bootfile.py tftp_dir mac cifs_server share directory image_to_restor cifs_username cifs_password ip netmask gateway" | ||
| print("Usage: prepare_tftp_bootfile.py tftp_dir mac cifs_server share directory image_to_restor cifs_username cifs_password ip netmask gateway") |
|
Dear @vishesh92, just for clarification. The title says Ubuntu and the description says Debian. Which ones is this fix for? Or is it for the Debian-based, that includes Ubuntu? |
The packages are same for both Ubuntu and Debian. |
vishesh92
changed the title
|
@vishesh92 |
I couldn't confirm if some of the scripts works as expected before and after changes. Should we rely on just BO for this? @blueorangutan package |
|
@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18424 |
|
[SF] Trillian Build Failed (tid-16503) |
|
@blueorangutan test ubuntu24 xenserver-84 |
|
@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu24 mgmt + xenserver-84) has been kicked to run smoke tests |
|
[SF] Trillian test result (tid-16504)
|
|
@blueorangutan test ubuntu22 kvm-ubuntu22 |
|
@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu22 mgmt + kvm-ubuntu22) has been kicked to run smoke tests |
|
[SF] Trillian test result (tid-16540)
|
|
@vishesh92 |
|
this fixes the issues with cloudstack installation on ubuntu 22 |
|
[SF] Trillian Build Failed (tid-16545) |
|
[SF] Trillian Build Failed (tid-16546) |
| print ("Copying new private key file as it is not matching with old file") | ||
| shutil.copyfile(newKey, currDir + pathSep + "id_rsa.cloud") | ||
| os.chmod(currDir + pathSep + "id_rsa.cloud", 0644) | ||
| os.chmod(currDir + pathSep + "id_rsa.cloud", 0o644) |
| import os, re, sys | ||
| import xml.dom.minidom | ||
| import xmlrpclib | ||
| import xmlrpc.client |
| util.SMlog("scanning2 (target=%s)" % target) | ||
| dom = nfs.scan_exports(target) | ||
| print >>sys.stderr,dom.toprettyxml() | ||
| print(dom.toprettyxml(), file=sys.stderr) |
| util.SMlog("scanning2 (target=%s)" % target) | ||
| dom = nfs.scan_exports(target) | ||
| print >>sys.stderr,dom.toprettyxml() | ||
| print(dom.toprettyxml(), file=sys.stderr) |
| util.SMlog("scanning2 (target=%s)" % target) | ||
| dom = nfs.scan_exports(target) | ||
| print >>sys.stderr,dom.toprettyxml() | ||
| print(dom.toprettyxml(), file=sys.stderr) |
| util.SMlog("scanning2 (target=%s)" % target) | ||
| dom = nfs.scan_exports(target) | ||
| print >>sys.stderr,dom.toprettyxml() | ||
| print(dom.toprettyxml(), file=sys.stderr) |
| syslog.syslog(s) | ||
| else: | ||
| print >>sys.stderr, s | ||
| print(s, file=sys.stderr) |
| def get_pif_by_uuid(self, uuid): | ||
| pifs = map(lambda (ref,rec): ref, | ||
| filter(lambda (ref,rec): uuid == rec['uuid'], | ||
| self.__pifs.items())) | ||
| pifs = [ref_rec3[0] for ref_rec3 in [ref_rec for ref_rec in list(self.__pifs.items()) if uuid == ref_rec[1]['uuid']]] |
| def get_pifs_by_device(self, device): | ||
| return map(lambda (ref,rec): ref, | ||
| filter(lambda (ref,rec): rec['device'] == device, | ||
| self.__pifs.items())) | ||
| return [ref_rec4[0] for ref_rec4 in [ref_rec1 for ref_rec1 in list(self.__pifs.items()) if ref_rec1[1]['device'] == device]] | ||
|
|
||
| def get_pif_by_bridge(self, bridge): | ||
| networks = map(lambda (ref,rec): ref, | ||
| filter(lambda (ref,rec): rec['bridge'] == bridge, | ||
| self.__networks.items())) | ||
| networks = [ref_rec5[0] for ref_rec5 in [ref_rec2 for ref_rec2 in list(self.__networks.items()) if ref_rec2[1]['bridge'] == bridge]] |
|
|
||
| from xen.util.xmlrpcclient import ServerProxy | ||
| from xmlrpclib import Error | ||
| from xmlrpc.client import Error |
|
@blueorangutan package |
|
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18554 |
|
@blueorangutan test ubuntu24 xcpng82 |
|
@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu24 mgmt + xcpng82) has been kicked to run smoke tests |
|
@vishesh92 I think we do not need to update the xen/xcpng scripts, what do you think ? |
|
discussed with @vishesh92 |
|
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch. |
Description
This PR fixes package installation errors on Ubuntu and debian
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
How did you try to break this feature and the system with this change?