KBEC-00515 - Potential issues when upgrading from ec-perl 5.8 to cb-perl 5.32

Description

CD/RO has removed support for Perl 5.8 in v2023.12.0 and now supports Perl 5.32. This change has been made in order to resolve a security issue. Due to this change, you may need to update your existing Perl scripts to run on the new version. This article will cover a number of potential issues that you could encounter due to the upgrade and how to fix them.

There are three major types of issues that you could encounter:

Programming perl in a non-recommended way and using syntax construction in a non-supported way
Perl module incompatibility due to major version upgrade
Using perl features that are no longer available or changed between versions

Issues:

1) Call defined on arrays or hashes

The following code defined(@array) or defined(%hash) was deprecated since perl 5.8.9.

Let’s consider the following code:

use strict;
use warnings;

# create an array (NOT THE REFERENCE)
my @array = (1, 2, 3);
# create a hash (AGAIN, NOT THE REFERENCE)
my %hash = (one => 'two', three=> 'four', five => 'six');

# try to call defined on @array
if (defined(@array)) {
    print("ARRAY Defined!\n");
}

# try to call defined on %hash
if (defined(%hash)) {
    print("HASH Defined!\n");
}

On 5.8.9 it throws a warning:

Perl version is: 5.008009
defined(@array) is deprecated at defined.pl line 13.
	(Maybe you should just omit the defined()?)
defined(%hash) is deprecated at defined.pl line 17.
	(Maybe you should just omit the defined()?)
ARRAY Defined!
HASH Defined!

But on 5.32 it is a FATAL:

Perl version is: 5.032001
Can't use 'defined(@array)' (Maybe you should just omit the defined()?) at defined.pl line

Solution:

Follow the interpreters suggestions and omit the defined:

BEGIN {
    print "Perl version is: $]\n";
}
use strict;
use warnings;



my @array = (1, 2, 3);
my %hash = (one => 'two', three=> 'four', five => 'six');


if (@array) {
    print("ARRAY Defined!\n");
}

if (%hash) {
    print("HASH Defined!\n");
}

Also, notice that @array and %hash are objects, not references. For references it could be different. See this example of how to handle hash and array references properly:

BEGIN {
    print "Perl version is: $]\n";
}

use strict;
use warnings;

my $array = [1, 2, 3];
my $hash = {one => 'two', three=> 'four', five => 'six'};

my $empty_array = [];
my $empty_hash = {};

# improper
if ($empty_array) {
    print("[WRONG]: Empty array is TRUE\n");
}

# improper
if ($empty_hash) {
    print("[WRONG]: Empty hash is TRUE\n");
}

# improper
if (defined($empty_array)) {
    print ("[WRONG]: Empty array is defined\n");
}

# improper
if (defined($empty_hash)) {
     print("[WRONG]: Empty hash is defined\n")
}

# proper checks
if (@$array) {
    print("[CORRECT]: Array is NOT empty\n");
}

if (%$hash) {
    print("[CORRECT]: Hash is NOT empty\n");
}


if (!@$empty_array) {
    print("[CORRECT]: Array IS empty\n");
}

if (!%$empty_hash) {
    print("[CORRECT]: Hash IS empty\n");
}

Output of above script:

Perl version is: 5.032001
[WRONG]: Empty array is TRUE
[WRONG]: Empty hash is TRUE
[WRONG]: Empty array is defined
[WRONG]: Empty hash is defined
[CORRECT]: Array is NOT empty
[CORRECT]: Hash is NOT empty
[CORRECT]: Array IS empty
[CORRECT]: Hash IS empty

2) Varying order of hash keys

Starting from perl 5.18 hash keys order is not consistent. Let’s consider the following code:

BEGIN {
    print "Perl version is: $]\n";
}

use strict;
use warnings;
my $hash = {one => 'two', 'three'=> 'four', 'five' => 'six'};

print("=== Wrong case ===\n");
for my $k (keys %$hash) {
    print("$k: $hash->{$k}\n");
}

print("=== Correct case, with solution ===\n");

for my $k (sort(keys %$hash)) {
    print("$k: $hash->{$k}\n");
}

Using ec-perl (5.8) it always returns the following:

Perl version is: 5.008009
=== Wrong case ===
three: four
five: six
one: two
=== Correct case, with solution ===
five: six
one: two
three: four

Using cb-perl (5.32), the first output can be different. You cannot rely on the output being the same every time:

# 1st call:
Perl version is: 5.032001
=== Wrong case ===
three: four
five: six
one: two
=== Correct case, with solution ===
five: six
one: two
three: four

# 2nd call:
Perl version is: 5.032001
=== Wrong case ===
three: four
one: two
five: six
=== Correct case, with solution ===
five: six
one: two
three: four

As you can see in “wrong case” section it has different orders while in “correct case, with solution” section the order is always the same.

Solution:

Use sort function to have the same (alphabetical) order of hash keys.

3) LWP::UserAgent and insecure connections

Perl 5.8.9 (ec-perl) uses LWP::UserAgent version 5. Perl 5.32 (cb-perl) uses LWP::UserAgent version 6.

They are mostly compatible but there are few important differences. In LWP 5 if you need to disable SSL verification of hosts you have to set the following environment variable:

PERL_LWP_SSL_VERIFY_HOSTNAME to 0

Or in perl code:

$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;

This does not work for LWP 6. To make it work you need to specify a ssl_opts parameter. This is detailed on the following page (provided with link to particular section):

LWP::UserAgent - ssl_opts

4) Prototypes and attributes

Prototypes and attributes are two independent perl mechanics. These mechanics have a very limited use in practice and there are few reasons to use the prototypes. Prototypes are designed to bypass some perl syntax and change the rules of list context. See more details:

Perl prototypes

Attributes:

Perl attributes

The general rule of prototypes and attributes is not to use them. In most cases prototypes could be removed from subroutine definition and it will work without any changes.

With attributes, most perl programmers never face them in their whole career. It is very hard to give any advice on them except “change your code and remove attributes”. In general attributes should be working fine. For the most part, they are utilizing the same syntax and the same properties for both 5.8 and 5.32.

5) Use threads

It is not recommended to use threads in perl. Threads are considered to be unstable and therefore removed from perl distribution by default. However, at the time of this writing, cb-perl is compiled with threads support.

It may lead to some unexpected behaviour and hard-to-fix bugs in your code/env. To avoid that consider one of the following approaches:

Make your code not depend on threads and use another mechanism of parallel execution.
Try to switch to use forks for linux environments.

6) Smart match

Smart match operator: ~~

It is not recommended to use smart match in perl. It has an overburden logic and is very slow. Its behaviour is very complex and it is hard to predict how it will work on certain versions of perl.

See for more details: smartmatch ~~ - Don’t use it! And perldoc (perlop): perlop - Smart match

7) Using . in default INC path

Starting from v5.26, perl does not have '.' as a valid library include path.

Perl has all its include path in a special variable named @INC. Let’s see the following example (no '.' in INC for 5.32).

$> cb-perl -e 'print $],"\n";print join "\n", @INC, "\n"'
5.032001
/opt/electriccloud/electriccommander/perl/lib/site_perl/5.32.1/darwin-thread-multi-2level
/opt/electriccloud/electriccommander/perl/lib/site_perl/5.32.1
/opt/electriccloud/electriccommander/perl/lib/5.32.1/darwin-thread-multi-2level
/opt/electriccloud/electriccommander/perl/lib/5.32.1

And for ec-perl (5.8):

ec-perl -e 'print $],"\n";print join "\n", @INC, "\n"'
5.008009
/opt/electriccloud/electriccommander/perl-legacy/lib/5.8.9/darwin-thread-multi-2level
/opt/electriccloud/electriccommander/perl-legacy/lib/5.8.9
/opt/electriccloud/electriccommander/perl-legacy/lib
/opt/electriccloud/electriccommander/perl-legacy/lib/site_perl/5.8.9/darwin-thread-multi-2level
/opt/electriccloud/electriccommander/perl-legacy/lib/site_perl/5.8.9
/opt/electriccloud/electriccommander/perl-legacy/lib/site_perl
/Users/build/ec-toolchain/out/x86_64_Darwin/perl-5.8.9/lib/5.8.9/darwin-thread-multi-2level
/Users/build/ec-toolchain/out/x86_64_Darwin/perl-5.8.9/lib/5.8.9
/Users/build/ec-toolchain/out/x86_64_Darwin/perl-5.8.9/lib/site_perl/5.8.9/darwin-thread-multi-2level
/Users/build/ec-toolchain/out/x86_64_Darwin/perl-5.8.9/lib/site_perl/5.8.9
.

Note: cb-perl has a lower count of include paths since it is built with relocatable inc flag enabled.

Let’s consider the following directory structure with the following code:

$> ls
Demo.pm test.pl

# Demo.pm
package Demo;

use strict;
use warnings;

sub test {
    print "TEST SUB\n";
}
1;

# test.pl

use strict;
use warnings;
use Demo;

Demo::test();

Now if you run test.pl from this current directory with those 2 files, using ec-perl (5.8), you will get everything right in place. It will print “TEST SUB”.

But for cb-perl (5.32) the situation is different:

Can't locate Demo.pm in @INC (you may need to install the Demo module) (@INC contains: /opt/electriccloud/electriccommander/perl/lib/site_perl/5.32.1/darwin-thread-multi-2level /opt/electriccloud/electriccommander/perl/lib/site_perl/5.32.1 /opt/electriccloud/electriccommander/perl/lib/5.32.1/darwin-thread-multi-2level /opt/electriccloud/electriccommander/perl/lib/5.32.1) at test.pl line 3.
BEGIN failed--compilation aborted at test.pl line 3.

Solution:

There are 2 ways of fixing this issue:

Set an environment variable PERL5LIB.
Use the 'use lib' pragma.

Use PERL5LIB env var

PERL5LIB="$PERL5LIB:." cb-perl test.pl

Use this approach when you want to avoid code modifications. Also please note, we are appending a dot to the PERL5LIB env var instead of PERL5LIB='.' because it may have some values already and we do not want to lose them. : is used as a UNIX path separator.

Use lib pragma

Modify the test.pl file to be:

# adding use lib pragma usage
use lib '.';
use strict;
use warnings;
use Demo;

Demo::test();

Now it will work because use lib appends value to the @INC.

8) UTF8 issues and their friends

Unicode issues are difficult in any programming language. Perl has comprehensive support of UTF8 but on a bit lower lever than people are expecting. There is no particular advice on how to deal with that, but the sign of having unicode issues with perl is the following warning:

Wide character in <SUB>. The reasons:

how to get rid of Wide character in print at?

"Wide character in print"

To get an understanding where to go and how to mitigate an issue you may want to get started from the note in JSON::XS module by Marc Lehmann.

JSON::XS

9) Pseudo hashes usage

Pseudo-hashes deprecated

Why do I get "Pseudo-hashes are deprecated"?

10) Using a hash or an array as a reference are now fatal errors

For example, %foo->{"bar"} now causes a fatal compilation error. These have been deprecated since before v5.8, and have raised deprecation warnings since then.

11) scalar(%hash) return signature changed

The value returned for scalar(%hash) will no longer show information about the buckets allocated in the hash. It will simply return the count of used keys. It is thus equivalent to 0+keys(%hash).

A form of backward compatibility is provided via Hash::Util::bucket_ratio() which provides the same behavior as scalar(%hash) provided in Perl 5.24 and earlier.

12) switch module was removed

The switch module is no longer available by default in Perl 5.32, and is not present in our distribution. As a workaround, replace switch occurrences with if/elsif or given/when. For example:

switch ($result) {
  case("success") {
    exit 0;
  }
  case("failed") {
    exit 1;
  }

Using if/else:

if ($result eq "success") {
  exit 0;
}
elsif ($result eq "failed") {
  exit 1;
}

Using given/when:

given ($result) {
  when ("success") {
    exit 0;
  }
  when ("failed") {
    exit 1;
  }
}