compiling from py2 to py3; enable tcoffee aligner, iqtree #442

dengzq1234 · 2020-03-26T12:09:05Z

python 3.x compilation:
- remove deprecated 'U' mode
- raw_input() -> input()
- print()
ete_build tools:
- enable experimental tcoffee aligner
- fix bug of iqtree

jhcepas · 2020-03-26T14:48:57Z

VERSION

could we stick to 3 digits versioning?
I tried to used the following convention: if version is A.B.C, A=Major changes (w/ potential compatibility problems). B=Add on features. C=Bug fixes and small changes

jhcepas · 2020-03-26T14:49:19Z

ete3/evol/evoltree.py

@@ -126,7 +126,8 @@ def __init__(self, newick=None, alignment=None, alg_format="fasta",
        self.workdir = '/tmp/ete3-tmp/'
        if not binpath:
            ete3_path = which("ete3")
-            binpath = os.path.join(os.path.split(ete3_path)[0], "ete3_apps", "bin")
+            #binpath = os.path.join(os.path.split(ete3_path)[0], "ete3_apps", "bin") #Ziqi


remove commented line

jhcepas · 2020-03-26T14:50:37Z

ete3/tools/ete.py

+
+            if ete3_path:
+                builtin_apps_path = os.path.join(os.path.split(ete3_path)[0], "ete3_apps/bin") #Ziqi
+                #builtin_apps_path = os.path.split(ete3_path)[0]


clean up commented code

jhcepas · 2020-03-26T14:51:14Z

ete3/tools/ete_build_lib/apps.py

@@ -135,7 +136,8 @@ def get_call(appname, apps_path, exec_path, cores):
    except KeyError:
        return None

-    bin_path = os.path.join(apps_path, "bin")
+    #bin_path = os.path.join(apps_path, "bin") 


remove comment

jhcepas · 2020-03-26T14:52:12Z

ete3/tools/ete_build_lib/task/cog_selector.py

        valid_cogs = sorted([sing for sing in all_singletons if len(sing) >= min_species],
-                            sort_cogs_by_size)
+                            sort_cogs_by_size) #Ziqi issue


I remember this line was linked to a bug in Python 3. Is it solved?

It's fixed now

jhcepas

All good Ziqi. just minor comments for clean up

jhcepas · 2020-03-26T14:53:51Z

ete3/tools/ete_build_lib/task/iqtree.py

-                    raise ConfigError("IQTREE CODON models require a codon alignmen.\nProvide nucleotide sequences with '-n' and set '--nt-switch-thr 0.0' to ensure codon alignments.")
+                #print(GLOBALS["seqtypes"])
+                #print(seqtype)
+                #if seqtype == "aa" or "aa" not in GLOBALS["seqtypes"]: #logic confusion Ziqi


could you explain what was the issue here? did you check that your version works as expected?

jhcepas · 2020-03-26T14:54:45Z

ete3/tools/ete_build_lib/task/tcoffee.py

@@ -16,7 +16,7 @@ def __init__(self, nodeid, multiseq_file, seqtype, conf, confname):
        self.confname = confname
        self.conf = conf
        # Initialize task
-        task_name =   "TCoffee(-%s)" %conf["-mode"] if "-mode" in conf else "TCoffee"
+        task_name =   "TCoffee(-%s)" %conf["-mode"] if "-mode" in conf else "TCoffee" #Ziqi


please remove all #Ziqi tags from final code.

jhcepas · 2020-03-26T14:55:27Z

utils/conda_build/meta.yaml.template

@@ -8,7 +8,8 @@ source:
    #git_url: https://github.com/jhcepas/ete.git

 build:
-   noarch_python: True
+    noarch: python
+    string: pyh39e3cac_0


why the string?

jhcepas · 2020-03-26T14:55:37Z

utils/release.py

@@ -1,5 +1,6 @@
 import re
-import commands
+#import commands


jhcepas · 2020-03-26T18:21:06Z

ete3/tools/ete_build.py

@@ -691,6 +695,7 @@ def _main(arguments, builtin_apps_path=None):
            # setup portable apps
            config = {}
            for k in apps.builtin_apps:
+                # concatnate the path with "bin"


concatenate

jhcepas · 2020-03-26T18:21:29Z

VERSION

please, add new line at the end

jhcepas · 2020-03-26T18:22:25Z

ete3/test/test_ete_build/test_modeltest.py

@@ -54,4 +54,4 @@ def test_01_aa_genetree_worflows_with_model_test(self):
                            SeqGroup(alg_trimmed)

 if __name__ == "__main__":
-    unittest.main()
+    unittest.main()


same here. add new line. not a big deal, but it is usually recommended

jhcepas · 2020-03-26T18:23:22Z

ete3/tools/ete_build.py

        if arguments[1] == "check":
            if not pexist(APPSPATH):
+                print('this is apps_path', APPSPATH)


this looks as a debug message. Remove or rephrase more formally

jhcepas · 2020-03-26T18:23:37Z

ete3/tools/ete_build.py

@@ -998,7 +1003,7 @@ def _main(arguments, builtin_apps_path=None):
    exec_group.add_argument("--nochecks", dest="nochecks",
                            action="store_true",
                            help="Skip basic checks (i.e. tools available) everytime the application starts.")
-
+    


remove empty chars

jhcepas · 2020-03-26T18:31:00Z

ete3/tools/ete_build_lib/task/iqtree.py

@@ -29,8 +29,8 @@ def __init__(self, nodeid, alg_phylip_file, constrain_id, model,
            base_args["-st"] = "AA" if seqtype == "aa" else "DNA"            
        else:
            if conf[confname]["-st"].startswith("CODON"):
-                if seqtype == "aa" or "aa" not in GLOBALS["seqtypes"]:
-                    raise ConfigError("IQTREE CODON models require a codon alignmen.\nProvide nucleotide sequences with '-n' and set '--nt-switch-thr 0.0' to ensure codon alignments.")
+                if seqtype == "aa" or "aa" in GLOBALS["seqtypes"]: 


I think it was right in the original version. If codon models are selected, we need to check that alg was done with aa and then translated back into nts.

So, here we expect seqtype = 'nt' and 'aa' in GLOBLALS.

if seqtype is aa or is nt but aa not in globals, raise the Error (which I think is what the IF is covering)

jhcepas · 2020-03-26T18:39:09Z

ete3/tools/ete_build_lib/utils.py

@@ -651,4 +651,4 @@ def cmp(x, y):
    """cmp() exists in Python 2 but was removed in Python 3.
    This implements the same behavior on both versions.
    """
-    return (x > y) - (x < y)
+    return (x > y) ^ (x < y)


??? this was working before. Why the change? the impact on other functions is high!

dengzq1234 added 5 commits February 13, 2020 16:05

fork from etetoolkit, use t_coffee aligner, iqtree

7f9c05a

remove release_3.1.18 package

57683c9

remove empty ete

349be7d

change the way to formulate path of ete3_apps

7989722

remove deprecated 'U' mode

7877b83

jhcepas reviewed Mar 26, 2020

View reviewed changes

jhcepas requested changes Mar 26, 2020

View reviewed changes

dengzq1234 added 3 commits March 26, 2020 18:03

clean lines; compile sorted() in python3

5de9723

undo fix

53b6498

clean lines

c054307

jhcepas reviewed Mar 26, 2020

View reviewed changes

VERSION Outdated

3.1.2

Copy link

Member

jhcepas Mar 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please, add new line at the end

jhcepas reviewed Mar 26, 2020

View reviewed changes

dengzq1234 added 2 commits March 26, 2020 23:45

change cmp(); add new line; clean comments

c6ec153

add new line

eb735a4

jhcepas merged commit b524047 into etetoolkit:master Mar 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

compiling from py2 to py3; enable tcoffee aligner, iqtree #442

compiling from py2 to py3; enable tcoffee aligner, iqtree #442

dengzq1234 commented Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

dengzq1234 Mar 26, 2020

jhcepas left a comment

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

jhcepas Mar 26, 2020

compiling from py2 to py3; enable tcoffee aligner, iqtree #442

compiling from py2 to py3; enable tcoffee aligner, iqtree #442

Conversation

dengzq1234 commented Mar 26, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jhcepas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment