You are here

log files

8 posts / 0 new
Last post
log files
#1

Hello,

Could you tell me please how can I save a log file (or direct output in a log file) in rosetta? Normally, I would do "utility -options > log.file".

But when I run /bin/remodel.linuxgccrelease @flag_missing_loops > log.file, I get something different from when I run /bin/remodel.linuxgccrelease @flag_missing_loops.

Perhaps, there is a simple way to controll log files but I cannot find it for now.

Thank you in advance!

Category: 
Post Situation: 
Fri, 2018-01-26 07:29
shadrinams

What does "different" mean?

Two likely cases I can think of:

1) Rosetta is not finishing, or particularly Rosetta is crashing with a seg fault.  The > tool blocks output to disk in 32 kb blocks on most systems, so if the parent process crashes, the last 32 kb worth of output evaporates.  If there's a error message you are expecting; that's why it's gone, the best solution for debugging is to run in screen or tmux or whatever instead of trying to capture with >

2) Your expected log is a mix of stdout and stderr.  If there are error messages you're not seeing, you just need to redirect them too.  The syntax for also redirecting stderr depends on which shell you are using: google will tell you what to do.  It's usually something like 'command $2>1 > myfile".  (I never remember myself, i just google it at need)

I am assuming your @options file is the same in both cases, of course.  The "tracer" system - particularly the -mute option - will drastically change what output you get.  If your options files are different, tell us that and we will debug down a different path.

Fri, 2018-01-26 10:22
smlewis

Thank you for your reply!

I suppose it is a variation of case 1.

If I do not use the > tool, calculation is finished in similar way it is shown in the corresponding tutorial. If I use the > tool, it gets interapted somehow. Please see input and output files below.

PS: Tutorial example works well with the command bin/remodel.linuxgccrelease @flag_missing_loops > log but I get the same problem when I run bin/remodel.linuxgccrelease @flag_missing_loops >log. I am not sure that it is related although.

My command (with the > tool):

bin/remodel.linuxgccrelease @flag_missing_loops > log

flag_missing_loops input file:

-in:file:s input_files/5flz_model1_0003_A-numb.pdb
-remodel:blueprint input_files/blueprint_A.remodel
-run:chain A
-remodel:num_trajectory 1 
-remodel:quick_and_dirty

-out:path:all output_files
-out:file:scorefile 5flz_A_missing_loops.sc


5flz_A_missing_loops_bad.sc output file (with the > tool) is attached:

SEQUENCE: 
SCORE: total_score description 
SCORE:       0.000 5flz_model1_0003_A-numb_0001

5flz_A_missing_loops.sc output file (without the > tool):

SEQUENCE: 
SCORE: total_score dslf_fa13    fa_atr    fa_dun   fa_elec fa_intra_rep fa_intra_sol_xover4              fa_rep              fa_sol hbond_bb_sc hbond_lr_bb    hbond_sc hbond_sr_bb lk_ball_wtd       omega     p_aa_pp pro_close rama_prepro         ref yhh_planarity description 
SCORE:    7761.628     0.000 -3871.727  1010.537  -942.894        9.867             158.135            8292.329            2837.787     -27.328     -21.240     -55.124    -405.485    -127.333      91.957       9.453   442.974     240.064     119.633         0.023 5flz_model1_0003_A-numb_0001


Output pdb files also look different. When I use the > tool, the added loop is broken.

The log_bad file (with the > tool) is attached.

Mon, 2018-01-29 21:21
shadrinams

The space shouldn't matter: '> log' and '>log' are the same to most shells.

If the log_bad_part2.txt file is the end of the log file when you redirect the tracer output, then things are completing successfully, as far as Rosetta is concerned.

There really shouldn't be any reason for Rosetta to behave differently w/r/t output or scorefiles when the tracer output is redirected versus when it's not. (Unless an issue with redirection causes the shell to kill Rosetta.) That redirection is handled by the shell, not Rosetta, so as far as Rosetta is concerned there should be no difference.

I'm guessing that the issue isn't really a with/without log redirection issue. Instead, I'm guessing it may be related to the state of the directory when things are being run. Try this: make two identical (fresh) copies of the running directory under different names. In one run with the logfile redirection. In the other run things without.  I'm guessing that if you do this, you'll see identical results in both directories. (Whether "good" or "bad" depends on what the state of the directory is.)  Be sure to compare the last few lines (the ones labeled "protocols.jd2.JobDistributor") in the to-terminal run versus the to-logfile run. (Also check if anything is getting printed to the terminal in the logfile-redirected case.)

Tue, 2018-01-30 08:03
rmoretti

In addition to Rocco's suggestion: your executable path implies you are running Rosetta from inside the Rosetta install.  I would encourage you to make a clean directory elsewhere to run in, and either correct the path to the Rosetta executable or just symlink it in the new directory.  There are a huge number of files already in Rosetta, and lots are already PDBs or scorefiles or silent files (at least in the testing system) - let's remove the possibility that the wrong files are getting used or examined.  (Seems unlikely from your flags, but it's a good idea in any case.)

Tue, 2018-01-30 09:27
smlewis

Thank you for your informative answers! I created a new directory for each calculation (out of the Rosetta directory) as you advised. But I still wait for my calclations to finish to confirm that it works for long runs. For short runs, it does help!!

 

Now, I try to use DDG_monomer and see some problems that could be similar to the remodel issue (therefore I post it here).

When I run the Low Resolution Protocol (/gs/project/ear-065-aa/rosetta_src_2017.45.59812_bundle_12nodes/main/source/bin/ddg_monomer.linuxgccrelease @flag_ddg_low > low_res.log) I get an empty log file in 80% of cases with no output files and no errors. In remaining 20% runs, I get the following error with no output again:

caught exception 

[ ERROR ] EXCN_utility_exit has been thrown from: src/core/conformation/Conformation.hh line: 502
ERROR: Error in core::conformation::Conformation::residue(): The sequence position requested was greater than the number of residues in the pose.

And I see very similar situation with the High Resolution Protocol. The calculations get interapted with the same error or with no error at all when the application starts to calculate mutants.

I checked my pdb file, it contains 1 to 445 residues (chain C) with no holes in its sequence. I might miss something else but I cannot think of what it can be.

Please see below my mut_file and flags.

Thank you!

mut_file.dat

360 C PIKAA DA
362 C PIKAA EF
445 C PIKAA D

flag_ddg_low

-in:file:s input_files/min_cst_0.5.5flz_C_0001.pdb 

-ddg::mut_file input_files/mut_file.dat 

-ddg:weight_file soft_rep_design 

-database /gs/project/ear-065-aa/rosetta_src_2017.45.59812_bundle_12nodes/main/database/ 

-fa_max_dis 9.0 

-ddg::iterations 50 

-ddg::dump_pdbs true 

#-ignore_unrecognized_res 

-ddg::local_opt_only true 

-ddg::suppress_checkpointing true 

-in::file::fullatom 

-ddg::mean true 

-ddg::min false 

-mute all 

-ddg::output_silent true 

-out:level 400

 

Fri, 2018-02-09 07:19
shadrinams

My first impression would be that you're attempting to use PDB numbering where Rosetta is expecting Pose numbering (i.e. starting at one, going up by one per residue, ignoring chains, gaps, etc.) But since you said your PDB is numbered starting at 1 and goinging up sequentially without gaps, that's unlikely to be it.

It could happen, though, that Rosetta may be ignoring/deleting some of the residues. If Rosetta is ignoring some residues (because their occupancy is zero or because they're missing too many backbone atoms, etc.) then you might not have all the residues present. What I might suggest is to run your protein through the `score_jd2` application with the options `-out:pdb` and `-out:file:renumber_pdb`. This should make a new file which reflects what Rosetta sees internally. It should show you if you're missing any residue and where, and you can proceed from there.

Fri, 2018-02-09 08:04
rmoretti

Hello rmoretti,

Thank you very much for your help!

I found what is wrong. My fault! I used a wrong format of mutation file (resfile instead of alternative one). The results that I obtained look weird to me although. I think it will be better to create a new post about them. 

 

Mon, 2018-02-12 11:47
shadrinams