[Closed] Quick help with a basic linux command

Do * need to use the * character?

i.e.
cd //server/Inbound/ && ls -t *BBC* > //server/Inbound/List_20220214.txt

Reckon you'll need wildcards around BBC. ls -t *BBC*

Also, that's a UNC pathname, are you attaching to a Samba share?

Also also,

Do you need to supply permissions somewhere? (I've never used Samba)

You’ll need the wildcards, but looks OK for bash.

Also also also,

You've CD'ed to the directory then running the command. So I don't think you'd need the full path, just the filename.

(I'm assuming of course that CDing to a remote SMB share just like any other directory is even valid, as I said I've never used it.)

It might be weird if you have any directories that match *BBC*

I understand about not needing a full path name - tick.
Wildcard * - tick.
UNC / Samba? - err I have heard UNC being mentioned. Need to do some more research.

So:
cd //server/Inbound/ && ls -t *BBC* > List_20220214.txt

Christ - my first google attempt bought up Lion King. Muppet!

You won't catch any filename starting with '.' using 'ls'. 'ls -a' will show them but doesn't seem to pick them out when using wildcards. Best I found was 'ls -a | grep BBC'.

If it matters to you goodle "globbing" eg https://mywiki.wooledge.org/glob

And how can I add a match by extension as well as the BBC

i.e. BBC anywhere in the name. Must end with .txt. So it will only list the first 2 files below.

aaaa_BBC_0000.txt
aaaa_BBC_1111.txt
aaaa_BBC_1111.csv
aaaa_BBC_0000.csv
aaaa_ITV_1111.txt
aaaa_ITV_1111.csv

You won’t catch any filename starting with ‘.’ using ‘ls’

ls .*BBC* *BBC*

or use find.

BBC anywhere in the name. Must end with .txt.

*BBC*.txt

* means "anything". So it's listing [anything]BBC[anything].txt here.

any particular reason to string the two commands together. I know its what most of us do in real life on the command line - but sometimes its easier debugging if you just have one command.

ls -t //server/Inbound/*BBC* > //server/Inbound/List_20220214.txt

should work...

ls -t /server/Inbound/*BBC*.txt > List_20220214.txt

Edit: beaten to it!

(although with the slight difference that mine will generate the output list in the current directory)

Sweet. Thanks very much all.

cd //server/Inbound/ && ls -t *BBC*.txt > List_20220214.txt

This is good fun actually but it does get crazy powerful/complex very quickly.

ls -t //server/Inbound/*BBC* > //server/Inbound/List_20220214.txt

This looks better!

Earl
Free Member

I understand about not needing a full path name – tick.
Wildcard * – tick.
UNC / Samba? – err I have heard UNC being mentioned. Need to do some more research.

So:
cd //server/Inbound/ && ls -t *BBC* > List_20220214.txt

That won't work, you will need to mount the SMB share to a normal UNIX directory, then do your ls and > to that location.

If you've heard UNC mentioned but not Samba,

Is the target a Windows machine?

From wherever I am

From different machines? Assuming the remote machine is Linux I'd likely SSH to it and run the command directly on the box.

Why don't you tell us what you're actually trying to achieve, then we can tell you how best to go about it? There's little point in you asking "recommend me a hammer?" if you're sitting there with a box of screws.

Sounds like everything is in that one directory, but you do need to make sure it's mounted first. Something like this should do you (assuming it's not already mounted):

mount -t cifs -o username=USER_NAME,password=PASSWORD //server/Inbound /mnt
cd /mnt ; ls -t *BBC*txt > List_20220214.txt
cd
umount /mnt

Lot's more elegant ways to do it, but that should get you what you need (assuming permissions etc are all OK)

ls .*BBC* *BBC*

That's what I like about Linux, even the most basic commands have so many nuances.

Every time I look up something on Stack overflow, the depth of detail in the answers just amazes me, everything has so many options / gotchas / nuances that you never notice 99% of the time.

Yeah, you have to explicitly include hidden files… because 99.9999% of the time you want them to be ignored/excluded.

It’s like concocting a spell unix 🙂

It’s like concocting a spell unix

My favourite ones are sed when you need to substitute '/' but you have to escape it with '\' so the command just looks like \/\/\/\//\/ but actually does something useful....

I’ve been doing stuff with docker containers,the Linux stuff is so much more fun than the windoze ones and smaller.
(Although must admit containers are just Uber cool,either sex)

Some random stuff I'll throw in:

now=$(date '+%Y%m%d-%H%M%S') # timestamp for filename
out="List_$now.txt"
srv="//server/Inbound" # server mountpoint
find $srv -maxdepth 1 -type f -iname '*BBC*' > $srv/Inbound/"$out"

Curious if generated file is processed further? So may not need to have the intermediary file in the first place? You can process output of find within a do/read/while loop - find is a better option for this than ls.

My favourite ones are sed when you need to substitute ‘/’ but you have to escape it with ‘\’

You can avoid that by using a different character in place of '/' to delimt the match/replacement patterns. Just pick something that isn't in your regex.

(base) M-000582:~ jabbott $ echo $HOME|sed 's/\//:/g'
:Users:jabbott
(base) M-000582:~ jabbott $ echo $HOME|sed 's|/|:|g'
:Users:jabbott

Far more readable!

That’s what I like about Linux, even the most basic commands have so many nuances.

Every time I look up something on Stack overflow, the depth of detail in the answers just amazes me, everything has so many options / gotchas / nuances that you never notice 99% of the time.

One of the things I don't like about it is that 99% of those answers don't work on whatever it is you've got in front of you. 😁

Things like 'find the files' are deceptively simple until they're not because of weird edgecases and filenames that only a maniac would think up 🤷‍♂️

Ok. I don't know the exact details and I'm not very clued up in this world.

My program needs to import into a db all the CSV files in a folder that match by wildcard and extensions. Date modified ASC. Hello 1980's.

My program will be running on linux with the folder given to me as a UNC path name (???)

My idea was instead of trying to read the folder a programing language - just run an os command from my program to do the heavy lifting and just read in the names of the CSV files to import - from this new temp txt file.

I assume if my program needs to run on a Windows machine, I will just need to write the equivalent os command in DOS/powershell?

Yes I'm out of my depth a bit. 99.99% of any coding I do is read from dB, manipulate, write to dB. Basic stuff.

When you say "import into a db" do you mean import the filenames/paths? Or reading the CSV files and generating a db based on their contents?

The latter is much more of an undertaking of course!

Other than that I agree, getting the shell to do the heavy lifting of the globbing sounds like a good plan. Might need to be a bit careful about things like line endings if you run on either/or Linux/Windows but that's not an insurmountable issue by any stretch.

Import the contents of the csv file, a bit of transforming, insert into a DB table. Been done a million times by people down the ages.

In terms of line endings, I hope the API (I think it's called fgets() from memory) will take care of all that. Well I'll find out soon!

Aye, I was thinking of the CSV that's all straight-forward until you reach the weird edgecases and field names dreamed up by a maniac 🤪

Uniform and consistent CSV fairly easy of course.

Anyway, good luck with your project 👍

Confused now. Your script earlier was supposed to be looking for .txt files and now you talk about importing csv files? Not that I am going to be much help 🙂

I don’t know the exact details

Then step away from the command line and find out. Otherwise you're pissing into the wind.

Where's the database held, is that the same remote machine? What has access to what? What OSes are involved? Is this a one-off, on-demand or periodically? What are you going to do when you have this list? How are you handling files already imported? What exactly are you responsible for, why is this your problem? Etc, etc, etc. And of course, the $64,000 question, WHY are you even doing this, what's barfing out CSVs in the first place?

There's a a hundred questions and a million variables here. You've got to define a problem accurately before you can ever hope to solve it.

sed when you need to substitute ‘/’

The delimiter is not fixed in sed, e.g. comma is used here and slash is converted to dot:

echo "a/b/c" | sed 's,/,.,g'

The delimiter is not fixed in sed,

Didn't know that!

Learn something new every day with Linux.

Pretty sure I've never seen a sed example not using '/' until this thread.....

Although back to hacking VBA today - finished the last bash project...

Apologies to all but the pedant in me just had a panic attack.

*BBC*.txt
* means “anything”. So it’s listing [anything]BBC[anything].txt here.

. is also a wildcard. * means any number of [anything] and . means a single instance of [anything] so you may as well do away with it in the above and just use *BBC*txt

srv=”//server/Inbound” # server mountpoint
find $srv -maxdepth 1 -type f -iname ‘*BBC*’ > $srv/Inbound/”$out”

You've already included Inbound when you declared the $srv variable, and you shouldn't need double quotes (depends on shell I guess)

find is a better option for this than ls.

Disagree - find is an excellent tool, but ls is much simpler, and if that does what you need why overcomplicate things?

Pretty sure I’ve never seen a sed example not using ‘/’ until this thread

Most versions of sed will automatically use the character following the s as a delimiter

I do agree that more info is required before the problem can be answered properly, but from what the OP has described so far I would say this would do you:

cd //server/Inbound/ ; ls -t *BBC*txt > List_20220214.txt

Just make sure the path is correct and mounted (sounds like that part is out of your control).

One of the things I don’t like about it is that 99% of those answers don’t work on whatever it is you’ve got in front of you.

A lot of the systems I'm on are embedded and run a bespoke busy box implementation of Linux, so limited command set and the commands you do have will only run a subset of normal options e.g. there's no find command but you can run "ls -R | grep".

Apologies to all but the pedant in me just had a panic attack.

*BBC*.txt
* means “anything”. So it’s listing [anything]BBC[anything].txt here.

. is also a wildcard. * means any number of [anything] and . means a single instance of [anything] so you may as well do away with it in the above and just use *BBC*txt

Well - if pedantry is your thing, I've been biting my knuckles all through this.

Yes and No. '.' is a regex character - but not in bash command lines.

So *.txt will be only files ending in .txt
*txt is anything ending in txt, like bbcwasheretxt

Every day is a school day!

My idea was instead of trying to read the folder a programing language – just run an os command from my program to do the heavy lifting and just read in the names of the CSV files to import – from this new temp txt file.

I assume if my program needs to run on a Windows machine, I will just need to write the equivalent os command in DOS/powershell?

Yes I’m out of my depth a bit. 99.99% of any coding I do is read from dB, manipulate, write to dB. Basic stuff.

Not sure which language you are using - but Python would make that trivial and could easily handle switching between OS if you are careful with the commands. Other languages may not be quite so nice for this sort of thing. I find python very useful for exactly the sort of task you are thinking about, as its much more readable when in two years time someone asks you to change something / it breaks.