The environment for SQL Replication is 3 level:

Multiple Publisher (Changed default port in SQL local and Client)
Remote Distributor (Changed default port in SQL local and Client)
Multiple Subscriber (Changed default port in SQL local and Client)

When I set up Publisher and Distributor as one server (same SQL Instance), then the agent will succeed.
But when I set up Publisher as an Individual SQL Server and a Remote Distributor at the other SQL server. The SQL Replication will fail.

The message I received below:

Message: Named Pipes Provider: Could not open a connection to SQL Server [5]. 
Message: Cannot initialize the data source object of OLE DB provider "SQLNCLI11" for linked server "repl_distributor".