Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
Visit Stack Exchange
Stack Internal
Knowledge at work
Bring the best of human thought and AI automation together at your work.
Explore Stack InternalThe output format should look like: IDENTIFICATION, YEAR, MONTH CODE, PRECIPITATION FOR THAT MONTH ADB103884_Mly, 1989, 1, 123.56, 2, 56.23, 3, 58.9, 4, -99.99, 5, 6.9, 6, 48.2, 7, 89,1, 8, 85.3, 9, 98.1, 10, 190.2, 11, 283.9, 12, -99.99
IDENTIFICATION, YEAR, MONTH CODE, PRECIPITATION FOR THAT MONTH ADB103884_Mly, 1989, 1, 123.56, 2, 56.23, 3, 58.9, 4, -99.99, 5, 6.9, 6, 48.2, 7, 89,1, 8, 85.3, 9, 98.1, 10, 190.2, 11, 283.9, 12, -99.99 The output format should look like: IDENTIFICATION, YEAR, MONTH CODE, PRECIPITATION FOR THAT MONTH ADB103884_Mly, 1989, 1, 123.56, 2, 56.23, 3, 58.9, 4, -99.99, 5, 6.9, 6, 48.2, 7, 89,1, 8, 85.3, 9, 98.1, 10, 190.2, 11, 283.9, 12, -99.99
The output format should look like:
IDENTIFICATION, YEAR, MONTH CODE, PRECIPITATION FOR THAT MONTH ADB103884_Mly, 1989, 1, 123.56, 2, 56.23, 3, 58.9, 4, -99.99, 5, 6.9, 6, 48.2, 7, 89,1, 8, 85.3, 9, 98.1, 10, 190.2, 11, 283.9, 12, -99.99 The output format should look like: IDENTIFICATION, YEAR, MONTH CODE, PRECIPITATION FOR THAT MONTH ADB103884_Mly, 1989, 1, 123.56, 2, 56.23, 3, 58.9, 4, -99.99, 5, 6.9, 6, 48.2, 7, 89,1, 8, 85.3, 9, 98.1, 10, 190.2, 11, 283.9, 12, -99.99
The output file can be called the year_mly.txt
If there are missing values for a specific month, it should be given a value of -99.99. This way, every ID has a value for each month of the specified year.
The output format should look like: IDENTIFICATION, YEAR, MONTH CODE, PRECIPITATION FOR THAT MONTH ADB103884_Mly, 1989, 1, 123.56, 2, 56.23, 3, 58.9, 4, -99.99, 5, 6.9, 6, 48.2, 7, 89,1, 8, 85.3, 9, 98.1, 10, 190.2, 11, 283.9, 12, -99.99
The output file can be called the year_mly.txt
If there are missing values for a specific month, it should be given a value of -99.99. This way, every ID has a value for each month of the specified year.
I have over 3000 files that need to be combined based on specific attributes in the files. For example, each file is named by its ID (Eg. 101567AD101567AD_Mly.txt).These IDs correspond to latitude and longitude values that I will concatenate to the file after. I need to add a leading column in each output file with the ID. I would then like to merge lines in the input files so that the output looks like IDENTIFICATION, YEAR, JANUARY MONTH CODE (1), JANUARY PRECIPITATION, FEBRUARY MONTH CODE (2), FEBRUARY PRECIPITATION, MARCH MONTH CODE (3), MARCH PRECIPITATION......
Currently, the files are separating each month into a new line like the example below where it is currently structured as YEAR, MONTH, UNKNOWN VALUE, PRECIPITATION AMOUNT.I would like to combine the 3000+ files based on the ID for each, which is their filename. Ideally, I would have a file for each year of data (1979,1980, 1981, etc) so that all files with those years of data (eg. 1981) would all be in one file with each line representing a different ID. The filenames describe the identification of where this data came from so I would like it as the leading column in each file. Here is an example of what the files look like:
YEAR, MONTH, REMOVE THIRD COLUMN, PRECIPITATION 1980 1 0 112.106 1980 2 0 131.909 1980 3 0 58.842 1980 4 0 42.075 1980 5 0 45.268 1980 6 0 126.168 1980 7 0 30.159 1980 8 0 32.168 1980 9 0 39.48 1980 10 0 46.117 1980 11 0 234.089 1980 12 0 180.227 1981 1 0 62.795 1981 2 0 131.817 1981 3 0 73.429 1981 4 0 77.282 1981 5 0 54.224 1981 6 0 81.217 1981 7 0 18.469 1981 8 0 12.695 1981 9 0 83.4 1981 10 0 130.957 1981 11 0 151.07 1981 12 0 190.582 1982 1 0 210.604 1982 2 0 143.732 1982 3 0 26.124 1982 4 0 45.952 1982 5 0 10.38 1982 6 0 33.602 1982 7 0 42.218 1982 8 0 9.541 1982 9 0 39.356 1982 10 0 87.037 The third column can just be removed. Any help to combine all these using perl or python would be great. Either a one-liner or script. I am using linux to do this.
The guy who used to do this work and based everything around Perl has retired leaving a lot of gaps in the knowledge transfer. I am struggling to piece some of it together.
I have over 3000 files that need to be combined based on specific attributes in the files. For example, each file is named by its ID (Eg. 101567AD).These IDs correspond to latitude and longitude values that I will concatenate to the file after. I need to add a leading column in each output file with the ID. I would then like to merge lines in the input files so that the output looks like IDENTIFICATION, YEAR, JANUARY MONTH CODE (1), JANUARY PRECIPITATION, FEBRUARY MONTH CODE (2), FEBRUARY PRECIPITATION, MARCH MONTH CODE (3), MARCH PRECIPITATION......
Currently, the files are separating each month into a new line like the example below where it is currently structured as YEAR, MONTH, UNKNOWN VALUE, PRECIPITATION AMOUNT.I would like to combine the 3000+ files based on the ID for each, which is their filename. Ideally, I would have a file for each year of data (1979,1980, 1981, etc) so that all files with those years of data (eg. 1981) would all be in one file with each line representing a different ID. The filenames describe the identification of where this data came from so I would like it as the leading column in each file. Here is an example of what the files look like:
YEAR, MONTH, REMOVE THIRD COLUMN, PRECIPITATION 1980 1 0 112.106 1980 2 0 131.909 1980 3 0 58.842 1980 4 0 42.075 1980 5 0 45.268 1980 6 0 126.168 1980 7 0 30.159 1980 8 0 32.168 1980 9 0 39.48 1980 10 0 46.117 1980 11 0 234.089 1980 12 0 180.227 1981 1 0 62.795 1981 2 0 131.817 1981 3 0 73.429 1981 4 0 77.282 1981 5 0 54.224 1981 6 0 81.217 1981 7 0 18.469 1981 8 0 12.695 1981 9 0 83.4 1981 10 0 130.957 1981 11 0 151.07 1981 12 0 190.582 1982 1 0 210.604 1982 2 0 143.732 1982 3 0 26.124 1982 4 0 45.952 1982 5 0 10.38 1982 6 0 33.602 1982 7 0 42.218 1982 8 0 9.541 1982 9 0 39.356 1982 10 0 87.037 The third column can just be removed. Any help to combine all these using perl or python would be great. Either a one-liner or script. I am using linux to do this.
The guy who used to do this work and based everything around Perl has retired leaving a lot of gaps in the knowledge transfer. I am struggling to piece some of it together.
I have over 3000 files that need to be combined based on specific attributes in the files. For example, each file is named by its ID (Eg. 101567AD_Mly.txt).These IDs correspond to latitude and longitude values that I will concatenate to the file after. I need to add a leading column in each output file with the ID. I would then like to merge lines in the input files so that the output looks like IDENTIFICATION, YEAR, JANUARY MONTH CODE (1), JANUARY PRECIPITATION, FEBRUARY MONTH CODE (2), FEBRUARY PRECIPITATION, MARCH MONTH CODE (3), MARCH PRECIPITATION......
Currently, the files are separating each month into a new line like the example below where it is currently structured as YEAR, MONTH, UNKNOWN VALUE, PRECIPITATION AMOUNT.I would like to combine the 3000+ files based on the ID for each, which is their filename. Ideally, I would have a file for each year of data (1979,1980, 1981, etc) so that all files with those years of data (eg. 1981) would all be in one file with each line representing a different ID. The filenames describe the identification of where this data came from so I would like it as the leading column in each file. Here is an example of what the files look like:
YEAR, MONTH, REMOVE THIRD COLUMN, PRECIPITATION 1980 1 0 112.106 1980 2 0 131.909 1980 3 0 58.842 1980 4 0 42.075 1980 5 0 45.268 1980 6 0 126.168 1980 7 0 30.159 1980 8 0 32.168 1980 9 0 39.48 1980 10 0 46.117 1980 11 0 234.089 1980 12 0 180.227 1981 1 0 62.795 1981 2 0 131.817 1981 3 0 73.429 1981 4 0 77.282 1981 5 0 54.224 1981 6 0 81.217 1981 7 0 18.469 1981 8 0 12.695 1981 9 0 83.4 1981 10 0 130.957 1981 11 0 151.07 1981 12 0 190.582 1982 1 0 210.604 1982 2 0 143.732 1982 3 0 26.124 1982 4 0 45.952 1982 5 0 10.38 1982 6 0 33.602 1982 7 0 42.218 1982 8 0 9.541 1982 9 0 39.356 1982 10 0 87.037 The third column can just be removed. Any help to combine all these using perl or python would be great. Either a one-liner or script. I am using linux to do this.
The guy who used to do this work and based everything around Perl has retired leaving a lot of gaps in the knowledge transfer. I am struggling to piece some of it together.
