I have POF setup of an enterprise server (IBM x3550m4) running with just an Intel PCIe-card-form-factor p3600 1.6Tb, hosting an ERP with MS SQL DB and a fileserver with ~400Gb of regular office files. Have done several benchmarks in the last months and everything is running just fine.

I will soon put this stuff in production, and I have a spare (new, with 1.2 Tb of 10k spindles) x3550m4... within my budget (~1500 euro, less is better), I’m thinking about two architectural pattern to get better reliability than the POF single node:

• Buy another enterprise SSD (maybe the samsung 1725a, 1.6 Tb too) for the other node, setup a SQL replication for the DB and fileserver sync daemon (DRFS, Synchthing, ecc), and failover the DNS in case of the first node fail (in any way). We can tolerate some hours of downtime;
• Buy a couple of smaller SAS SSD and RAID 1 them, use those as the primary storage, use the p3600 or the spindles for the replicas.

A couple of consideration about that:

• I think that today’s enterprise-class PCIe SSD in the first 5 years from deployment and with the right overprovisioning (like the ones that I mentioned) almost as reliable as a RAID controller, because they have a full solid-state storage controller, no moving parts and a declared MTBF that is very reassuring;
• Those services can tolerate even a day of downtime every 3-5 years without major impact on revenues;
• I don’t see the point of RAID and in general of node-level reliability if I can rely on better application clustering, this idea was inspired by hyperscalers/opencompute machines, that works on single PSU etc. because their reliability is achieved at an higher level.

What do you think about it? Any hints are welcome!

I wrote a little post about my messing around with the new Ubuntu, containers and OpenVSwitch.
I hope that could be useful to someone!

http://www.francescoprovino.com/2018/04/27/network-management-with-lxd-and-openvswitch-in-ubuntu-18-04/

@coliver said in Fiber VS Copper: racks interconnection in a pre-existing environment:

@francesco-provino said in Fiber VS Copper: racks interconnection in a pre-existing environment:

@scottalanmiller said in Fiber VS Copper: racks interconnection in a pre-existing environment:

@brandon220 said in Fiber VS Copper: racks interconnection in a pre-existing environment:

I do this both ways but usually lean towards fiber when I can't stay far enough away from electrical conduits. A major factor with fiber is that it is immune to noise from electrical circuits and can be run very close to them. I think fiber is cheap and I don't mind working with it. Yes, you can run 10G over CAT6 and it will also work well but you are limited on distance.

Fiber is really nice, any idea what the real cost difference is going to be?

Yes, 40 euro for 4 strands of 65m preterminated, tested, labeled and joined togeter in a single jacket of 6mm.

That's pretty inexpensive! Modules might be a bit pricier. Just make sure you clean the ends.

Modules are 5.80€ each.

With pulling mesh!

@scottalanmiller like that.

@scottalanmiller said in Fiber VS Copper: racks interconnection in a pre-existing environment:

@brandon220 said in Fiber VS Copper: racks interconnection in a pre-existing environment:

I do this both ways but usually lean towards fiber when I can't stay far enough away from electrical conduits. A major factor with fiber is that it is immune to noise from electrical circuits and can be run very close to them. I think fiber is cheap and I don't mind working with it. Yes, you can run 10G over CAT6 and it will also work well but you are limited on distance.

Fiber is really nice, any idea what the real cost difference is going to be?

Yes, 40 euro for 4 strands of 65m preterminated, tested, labeled and joined togeter in a single jacket of 6mm.

@wls-itguy said in Fiber VS Copper: racks interconnection in a pre-existing environment:

@bnrstnr said in Fiber VS Copper: racks interconnection in a pre-existing environment:

@wls-itguy said in Fiber VS Copper: racks interconnection in a pre-existing environment:

if speed is a factor, but when isn't it?

IDK somebody was posting about replacing a bunch of 10/100 switches the other day lol Fiber probably wont be on his radar for at least another 15 years

I think it was the same guy

Who?

@brandon220 said in Fiber VS Copper: racks interconnection in a pre-existing environment:

You are already at 300m too.

Just 65m in truth!

@gjacobse yes, we have machines like that. The production lines can draw up top 130kw of peak load.

@scottalanmiller
I don’t think fiber is much expensive, surely it’s cheaper for the bandwidth; if I will put something else like a backup server (or a new 4k nvr) in that room or if I repurpose it, I should be able to use that cable for absolutely EVERYTHING. AFAIK, today you can do more than 100Terabit on one strand of SMF, and 100Gbit is easily available… something like “the last cable you will ever need”.

Hi everybody, I run out of ports in a room (solid cat6 from the main rack) , so I have to add some more...
I have at least two different way to add them:

• run 6-10 new cat6 cables for 65 meters each from the main rack to the room;
• buy a new little wall rack (60€) for the room, run two cables (fiber or copper?) from the main rack to the new little rack of the room one and connect old and new room ports using the "old" cat6 pulled from the tubes (it's only 4 years old, perfect status).

I think that the solution with the satellite rack would be more clean (there are already too many cables in the main rack with servers etc) and future-proof, and the cost would be similar: 300m of cat6 costs ~90€ vs 40€ for pre-terminated 4-strands SMF + 60€ for the little rack.
I also think that running the backbone between racks in SMF would be much more future proof (either for bandwith and diameter) than another pair of copper... what do you think about it?

Completely different stuff.
SAN is not for file sharing, in any way.

We have very little space in the tubes. I’m thinking about going with MPO/MPT or multiple fiber cables for recabling, because they will take much less space.

EDIT: now I see that today a single SMF can do what an MPO did for a lot of bandwidth, uhm.

I have get rid of old 10/100 switches put on the floor just to make new ethernet ports available for two rooms of a small office with 3 desktop per room... two strategies:

• complete recabling of the room;
• just an AC AP for the desktops (the ports are enugh for VoIP deskphones/printers).

The thing is, recabling the rooms is hard because the cable tubes inside the walls are full so new cables will need new canalization etc... the WiFi can easily cost less (just the price of the AP + 1-2 AC pci adapters for the desktop that aren't WiFi enabled) and can reduce the cable clutting.
What do you think about it? I know cable is general more cost effective and reliable (and faster, of course), but this is a brownfield environment and one of the room will also need a new wall rack for the room switch, new patch panel...
Of course I will put one AP (ubiquiti lite) per room, and every AP will serve no more than 3 desktop + 3 mobile phones.

I know of many SMB that has purchased licenses like that for years, zero issues.

I'm in the EU, AFAIK it's perfectly legal. @matteo-nunziati , what do you think about it?

I usually use just plaintext filed nested in folders... other mainstream solutions?

@scottalanmiller said in SAN vs vSAN in a brownfield environment:

@francesco-provino said in SAN vs vSAN in a brownfield environment:

I've already pointed out that. The reply was that vSAN is a new/untested stuff, and they prefer to stick with a trusted route.

How are they defining this? vSAN is old and very trusted. SAN is older and has demonstrated that it is not to be trusted.

vSAN isn't actually old, it's the same age as SAN. The two are one and the same. Point that out, that vSAN and SAN aren't different things so all testing of one is of the other. Ask them point blank "You seem to be confused, I think you are recommending SAN only because you don't understand what vSAN is or means or you'd understand that you couldn't make that statement. I hope this means you are unknowledgeable of this subject matter and not just trying to scam us while thinking we are fools."

I think that the management never heard about vSAN, that's all.

There is the possibility that we will go with single SAN for two hosts before the other site is ready (maybe two years) .

@coliver said in SAN vs vSAN in a brownfield environment:

@scottalanmiller said in SAN vs vSAN in a brownfield environment:

@francesco-provino said in SAN vs vSAN in a brownfield environment:

@hobbit666 said in SAN vs vSAN in a brownfield environment:

@francesco-provino Why are they looking at changing?

Because of a licensing problem that took the ERP down after a CPU failure in a host. The license of a plugin was binded to a physical CPU. We are migrating away from this software in months, so it should not be an issue anymore.

How did that lead to this?

How would this new system resolve this issue either way? Wouldn't it still be CPU locked to a single host?

Exactly what I tried to point out.

@scottalanmiller said in SAN vs vSAN in a brownfield environment:

The salesman is trying to pull the "absolute answer con." In doing this, he takes the answer that someone got yesterday for their needs and pretends that they would have gotten the same answer if they were you, today.

If we were talking transportation, it would be like trying to convince a teenager today riding a bicycle into town that only idiots right bicycles and that all big business tycoons ride in horse drawn carriages.

This logic is based on... once upon a time when factors were different (different availability or horses, roads weren't smooth, bicycle tires were crap) that everyone used horses because it is what they had, and business people needed to stay dry was were pulled in carriages. The factors have changed, as have the answers. Horses aren't applicable to anyone today, and business people no longer use horse or carriages. Even as an individual in the 1850s, you'd likely have walked or ridden a horse, not had a carriage just for you.

So he's using two axis of trickery here, and really thinks the managers he is talking to are idiots.

What really matters from the enterprise standpoint is....

Good enterprises evaluate their needs and use the right technology to meet those needs. So what the salesman here is trying to suggest is, in fact, the exact opposite of what an enterprise would do. He's not only suggesting a final answer that is absurd and obviously wrong. He's also suggesting it based on trying to trick the managers into doing something emotional instead of logical - the opposite of enterprise.

I've already pointed out that. The reply was that vSAN is a new/untested stuff, and they prefer to stick with a trusted route.