[BUG] CI pipe line was failed #629

Activity

1. siosio (sio-funmatsu)

   added

   bug

   triageIssues that need to be looked atIssues that need to be looked at

   on Sep 20, 2024
2. maggiojoe2 commented on Dec 9, 2024

   

   maggiojoe2

   on Dec 9, 2024

   Last edited by maggiojoe2

   More actions

   Adding more context as I am having the same issue as of a few weeks ago. This is the error I am seeing when I enable debugging:

   ##[debug]found: '/usr/bin/sudo'
   ##[debug]/usr/bin/sudo arg: pip3 install iotedgedev==3.3.3
   ##[debug]/usr/bin/sudo arg: pip3 install iotedgedev==3.3.3
   ##[debug]   install
   ##[debug]   iotedgedev==3.3.3
   ##[debug]   iotedgedev==3.3.3
   ##[debug]
   ##[debug]ERROR: Could not find a version that satisfies the requirement iotedgedev==3.3.3 (from versions: 0.79.0, 0.80.0, 0.81.0, 0.82.0, 1.0.0, 1.1.0, 1.2.0, 1.3.0, 2.0.0, 2.0.1, 2.0.2, 2.1.0)
   ERROR: No matching distribution found for iotedgedev==3.3.3
   
   ##[debug]which 'iotedgedev'
   ##[debug]not found
   ##[debug]task result: Failed
   ##[error]Error: Unable to locate executable file: 'iotedgedev'. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.
   

   I am using ubuntu-latest for my vmImage. I have tried with other versions of ubuntu with no luck.

   Previously, I had manually installed iotedgedev with pip, but that is also throwing errors.
3. ko16a46 commented on Dec 17, 2024

   

   Brian Robles (ko16a46)

   on Dec 17, 2024

   Last edited by ko16a46

   More actions

   I ran into this issue as well. This is actually a bug observed when using the AzureIoTEdge@2 task in the Azure Pipeline Tasks repo, but it seems the root cause for the issue is the python dependency logic in this project

   Like maggiojoe2 I tried prior workarounds like using other versions of ubuntu, manually installing iotedgedev with pip, or setting the IOTEDGEDEV_VERSION to 2.1.0. All of these failed for me. Not to mention, the solution isn't ideal because we end up using an older version of iotedgedev

   In the end, I got it working by switching to windows-2022 as the vmImage, but this now causes issues with building edge modules on Windows. Some form of python dependency pinning needs to be done in this project perhaps, at least according to this guy's suggestions.
4. ko16a46 commented on Dec 17, 2024

   

   Brian Robles (ko16a46)

   on Dec 17, 2024

   More actions

   Update: I got it working by using the UsePythonVersion task to set the python version to 3.9 (which is apparently the latest version supported by iotedgehubdev which is a dependency of iotedgedev).

   Then I manually installed iotedgedev with a bash script pip3 install iotedgedev.
   The AzureIoTEdge task uses sudo pip3 install iotedgedev under the hood, but the sudo version of pip3 isn't affected by UsePythonVersion so the install fails. By manually installing iotedgedev after lowering the python version, the CI pipeline now works

   👍React with 👍2Reacted by maggiojoe2 and Sebastian Brandt
5. xjsxujingsong commented on Mar 20, 2025

   

   Jingsong Xu (xjsxujingsong)

   on Mar 20, 2025

   More actions

   Hi Brian Robles (@ko16a46) We used to use Ubuntu 20.04 to install iotedgedev for CI/CD pipeline, but yesterday, Microsoft mentioned that they will not support 20.04. We switched to 22.04 or 24.04 in which python version is higher than 3.9. We tried a few ways to use python 3.9 in 22.04 but failed. Anyone has a solution here?
6. ko16a46 commented on Mar 20, 2025

   

   Brian Robles (ko16a46)

   on Mar 20, 2025

   More actions

   Hi @ko16a46 We used to use Ubuntu 20.04 to install iotedgedev for CI/CD pipeline, but yesterday, Microsoft mentioned that they will not support 20.04. We switched to 22.04 or 24.04 in which python version is higher than 3.9. We tried a few ways to use python 3.9 in 22.04 but failed. Anyone has a solution here?

   In our case, we use ubuntu-22.04, and we use the UsePythonVersion@0 task. Something like below

   steps:
   - task: UsePythonVersion@0
     displayName: 'Setup python'
     inputs:
       versionSpec: '3.9'
       architecture: 'x64'
   
   - bash: |
       echo "Install iotedgedev"
       pip3 install iotedgedev
       iotedgedev --version
     displayName: 'Setup IoT Edge'

   Additionally, we couldn't use iotedgedev to build the modules, although in our case it was mainly due to trying to build cross-platform (the pipeline agent is amd64 but our edge modules target arm). We had to build our edge modules using docker build. For reference, we had something like the following:

   steps:
     // ...
     - bash: |
         # Based on answers in this Q&A link
         # https://learn.microsoft.com/en-gb/answers/questions/1387720/azure-iot-edge-image-build-for-amd64-arm64v8-in-az
   
         echo "Install binfmt (cross-platform emulator collection)"
         docker run --privileged --rm tonistiigi/binfmt --install all
         docker buildx create --name multiarchbuilder
         docker buildx use multiarchbuilder
         docker buildx inspect --bootstrap
         docker buildx install
         docker buildx ls
   
         # https://github.com/docker/buildx/issues/1170
         docker run --rm --privileged multiarch/qemu-user-static --reset -p yes -c yes
       displayName: 'Setup Docker'
   
     - task: Bash@3
       displayName: 'Build modules'
       inputs:
         filePath: scripts/build_modules.sh
         arguments: '-p arm64v8 -t $(Build.SourceVersion)'

   where build_modules.sh is a script we wrote to build modules similar to iotedgedev

   #!/bin/bash
   
   # Builds Azure IoT Edge modules in the modules folder
   #
   # Unlike `iotedgedev build`, this script correctly sets the target platform
   # so that modules can be cross-compiled for other architectures
   #
   # Usage:
   #   ./scripts/build_modules.sh [-p PLATFORM] [-t TAG]
   #
   # Options:
   #   -p  IoT Edge platform to build the image for. Defaults to arm64v8
   #   -t  Additional suffix tag to use for the built image
   
   set -e
   
   # Parse command line arguments
   PLATFORM="arm64v8"
   DOCKER_PLATFORM="linux/arm64"
   
   while getopts "p:t:" opt; do
     case $opt in
       t)
           TAG+=("$OPTARG")
           ;;
       p)
           PLATFORM=$OPTARG
           PLATFORM_BASE=${PLATFORM%%.*}
           case $PLATFORM_BASE in
             arm64v8)
               DOCKER_PLATFORM="linux/arm64"
               ;;
             amd64)
               DOCKER_PLATFORM="linux/amd64"
               ;;
             *)
               echo "Invalid platform: $PLATFORM_BASE" >&2
               exit 1
               ;;
           esac
           ;;
       \?)
         echo "Invalid option: -$OPTARG" >&2
         ;;
     esac
   done
   shift $((OPTIND -1))
   
   SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
   ROOT_DIR="$SCRIPT_DIR/.."
   pushd $ROOT_DIR
   
   echo "=================================================="
   echo "Building for platform: $PLATFORM ($DOCKER_PLATFORM)"
   echo "=================================================="
   echo ""
   
   for module in modules/*/module.json; do
       module_dir=$(dirname "$module")
       module_image=$(cat "$module_dir/module.json" | jq -r '.image.repository')
       module_version=$(cat "$module_dir/module.json" | jq -r '.image.tag.version')
       module_dockerfile=$(cat "$module_dir/module.json" | jq -r ".image.tag.platforms.\"$PLATFORM\"")
       module_file="$module_dir/$module_dockerfile"
   
       echo "----------------------------------------------------------------------------------------------------"
       echo "Building module: $module_image from $module_file"
       echo "----------------------------------------------------------------------------------------------------"
       echo ""
   
       if [[ ! -f $module_file ]]; then
           echo "No Dockerfile found: $module_file"
           continue
       fi
   
       docker build -f "$module_file" -t "$module_image" --platform "$DOCKER_PLATFORM" --load .
   
       for container_tag in "${TAG[@]}"; do
           module_tag="$module_image:$module_version-$container_tag-$PLATFORM"
           echo "Tagging image: $module_tag"
           docker tag "$module_image" "$module_tag"
       done
   
       echo ""
       echo ""
   done
   
   
   echo "=================================================="
   docker image ls
   
   popd

   After that, we can push modules, generate manifest, and deploy using the standard AzureIoTEdge@2 pipeline task

   👍React with 👍1Reacted by Jingsong Xu
7. xjsxujingsong commented on Apr 1, 2025

   

   Jingsong Xu (xjsxujingsong)

   on Apr 1, 2025

   More actions

   Brian Robles (@ko16a46) Thank you very much.